I like the use of the functional API here. I learned through a similar route and it was very helpful for me compared to trying to understand `torch.nn.Module`.
Here's a gist of my learning path if it's helpful to anyone: https://gist.github.com/kevmo314/294001659324429bae6749062a9...
Yes, these are two different learning paths. The detailed process learning is beneficial for future research, while the API-style approach is convenient and quick for getting started and using. Both are very useful!
I hadn't realized OpenAI's tiktoken Python library could work with other models outside of the OpenAI family, that's really useful: https://github.com/therealoliver/Deepdive-llama3-from-scratc...
It's more than just that, practically every notable open model released in the past year or so uses tiktoken as the tokenizer.
I'm glad to have helped you :)
great need; mulling over; shows up all the time in AI paradigms
[flagged]
[dead]