Lilian Weng@lilianweng · XAI 摘要
Thinky 可能要做的第一个产品是一整块旋钮面板，研究人员可以用它在训练过程中物理调节所有超参数。我们总有一天会做硬件，是时候了 😂
Probably the first product Thinky will build is a full panel of dials that researchers can use to physically adjust all the hparams during training. We gonna do hardware one day and it is the time 😂
Some teams use sweeps, heuristics, or scaling laws to determine their training LR. At Character, we just have Noam Shazeer dial it to the right value.