- A life ordinary
- Posts
- A life ordinary by Amit Sarkar - Issue #44
A life ordinary by Amit Sarkar - Issue #44
Amit Sarkar
February 16, 2024
Welcome to another edition of my newsletter.
Since the publishing of the last newsletter, I have walked a total of 39.89 kms as part of a new morning routine. I have done 4 strength training sessions, 2 Parkruns, 3 long runs totalling 41.12 kms and 1 yoga session. I have been trying to incorporate 2 runs, 2 strength training sessions, 2 long walks, and 1 yoga session per week. I aim to increase my strength while also maintaining decent levels of cardio fitness with flexibility and strength. Let’s see if I can keep up with my routine.
The other thing I have been obsessed about lately is the “use it or lose it“ concept. We all lose muscles, bones, and tissues (including brain tissue) as we age. And after being injured multiple times, I can safely say that our bodies are highly adaptable. The only caveat is we need to use it properly and regularly. We need to stay physically and mentally fit if we need to enjoy our old age. And to aid this we need to regularly exercise, have a proper diet and get good quality sleep. Three things I am now focussing on.
I also joined Vegan Runners UK. It’s the biggest club in Parkrun in the UK. I wanted to be associated with some club which represents my current lifestyle as well as my current passion. Hoping it brings more positive changes in my life.
Me and Rinat also recorded 2 podcasts and we are hoping to get those episodes out as soon as possible. Stay tuned.
I am also training for my first-ever Half Marathon event in Lisbon in March. The focus is on building running endurance, muscle strength, cardiovascular fitness and fueling strategy. I have completed 100 kms of running already this year. And hoping to complete another 50 kms or more before the event.
In this edition
🤖 Large Language Models (LLMs)
I recently attended a meetup where we discussed a paper on converting words into vectors. It was an unknown subject for me but the more I spent the evening reading the paper, the more curious I became. Everything we know about LLMs is a black box to most people.
During the meetup, a person recommended me to read a newsletter issue about LLMs
Some key points from the article are listed below. Please do read the whole article for better insight into the inner workings of LLMs.
Word vectors
Human beings represent English words with a sequence of letters, like C-A-T for cat. Language models use a long list of numbers called a word vector. For example, here’s one way to represent cat as a vector:
[0.0074, 0.0030, -0.0105, 0.0742, 0.0765, -0.0011, 0.0265, 0.0106, 0.0191, 0.0038, -0.0468, -0.0212, 0.0091, 0.0030, -0.0563, -0.0396, -0.0998, -0.0796, …, 0.0002]
Each word vector represents a point in an imaginary “word space,” and words with more similar meanings are placed closer together.
Words are too complex to represent in only two dimensions, so language models use vector spaces with hundreds or even thousands of dimensions.
A key advantage of representing words with vectors of real numbers (as opposed to a string of letters, like “C-A-T”) is that numbers enable operations that letters don’t.
Word meaning depends on context
When a word has two unrelated meanings, as with bank, linguists call them homonyms. When a word has two closely related meanings, as with magazine, linguists call it polysemy.
LLMs like ChatGPT are able to represent the same word with different vectors depending on the context in which that word appears. There’s a vector for bank (financial institution) and a different vector for bank (of a river). There’s a vector for magazine (physical publication) and another for magazine (organization). As you might expect, LLMs use more similar vectors for polysemous meanings than for homonymous meanings.
Word vectors provide a flexible way for language models to represent each word’s precise meaning in the context of a particular passage.
Transforming word vectors into word predictions
GPT-3, the model behind the original version of ChatGPT2, is organized into dozens of layers. Each layer takes a sequence of vectors as inputs—one vector for each word in the input text—and adds information to help clarify the meaning of that word and better predict which word might come next.
Each layer of an LLM is a transformer.
Research suggests that the first few layers focus on understanding the syntax of the sentence and resolving ambiguities. Later layers work to develop a high-level understanding of the passage as a whole.
The most powerful version of GPT-3 uses word vectors with 12,288 dimensions - that is, each word is represented by a list of 12,288 numbers. You can think of all those extra dimensions as a kind of “scratch space” that GPT-3 can use to write notes to itself about the context of each word. Notes made by earlier layers can be read and modified by later layers, allowing the model to gradually sharpen its understanding of the passage as a whole.
Can I have your attention please
The transformer has a two-step process for updating the hidden state for each word of the input passage:
In the attention step, words “look around” for other words that have relevant context and share information with one another.
In the feed-forward step, each word “thinks about” information gathered in previous attention steps and tries to predict the next word.
You can think of the attention mechanism as a matchmaking service for words. Each word makes a checklist (called a query vector) describing the characteristics of words it is looking for. Each word also makes a checklist (called a key vector) describing its own characteristics. The network compares each key vector to each query vector (by computing a dot product) to find the words that are the best match. Once it finds a match, it transfers information from the word that produced the key vector to the word that produced the query vector.
Each attention layer has several “attention heads,” which means that this information-swapping process happens several times (in parallel) at each layer. Each attention head focuses on a different task.
Attention heads frequently operate in sequence, with the results of an attention operation in one layer becoming an input for an attention head in a subsequent layer. Indeed, each of the tasks we just listed above could easily require several attention heads rather than just one.
The largest version of GPT-3 has 96 layers with 96 attention heads each, so GPT-3 performs 9,216 attention operations each time it predicts a new word.
The feed-forward step
After the attention heads transfer information between word vectors, there’s a feed-forward network that “thinks about” each word vector and tries to predict the next word. No information is exchanged between words at this stage: the feed-forward layer analyzes each word in isolation. However, the feed-forward layer does have access to any information that was previously copied by an attention head.
Attention heads retrieve information from earlier words in a prompt, whereas feed-forward layers enable language models to “remember” information that’s not in the prompt.
Indeed, one way to think about the feed-forward layers is as a database of information the model has learned from its training data. The earlier feed-forward layers are more likely to encode simple facts related to specific words, such as “Trump often comes after Donald.” Later layers encode more complex relationships like “add this vector to convert a country to its capital.”
How language models are trained
Many early machine learning algorithms required training examples to be hand-labeled by human beings. For example, training data might have been photos of dogs or cats with a human-supplied label (“dog” or “cat”) for each photo. The need for humans to label data made it difficult and expensive to create large enough data sets to train powerful models.
A key innovation of LLMs is that they don’t need explicitly labeled data. Instead, they learn by trying to predict the next word in ordinary passages of text. Almost any written material—from Wikipedia pages to news articles to computer code—is suitable for training these models.
Traditionally, a major challenge for building language models was figuring out the most useful way of representing different words—especially because the meanings of many words depend heavily on context. The next-word prediction approach allows researchers to sidestep this thorny theoretical puzzle by turning it into an empirical problem. It turns out that if we provide enough data and computing power, language models end up learning a lot about how human language works simply by figuring out how to best predict the next word. The downside is that we wind up with systems whose inner workings we don’t fully understand.
🤥 Lies You've Been Told
I regularly read Sahil Bloom’s newsletter and in one of his newsletters from last year, he mentioned all the lies we have ever been told. Be careful, as these are his opinions from his own experience. And what I have come to realize is that everyone has a different opinion of the same things. So feel free to absorb what connects with you the most.
These are the 23 lies discussed in the above article.
Money is the only type of wealth. - In reality, there are 5 types of wealth:
Financial (money)
Social (relationships)
Physical (health)
Mental (health, knowledge, faith)
Time (freedom)
The blind pursuit of financial wealth can rob you of the others. Never let that happen. Use incremental financial wealth as a tool to increase your other types of wealth.
Money can buy happiness. - Money is very directly correlated with happiness up to a baseline level of needs (food, shelter, etc.). Beyond that level, there are clear diminishing returns to the happiness derived from each incremental dollar.
There are timelines to achieve things in life. - The timelines we create for ourselves are mostly just arbitrary nonsense.
You don’t have to work hard. - What you work on is more important than how hard you work, but if you're striving for great things, you have to work hard.
You need to have formal mentors. - Instead of focusing on finding a formal mentor, build a Personal Board of Advisors. The key features of a Personal Board of Advisors:
Group of 5-10 individuals. The members don't need to know each other or that they are on your board. Informality is a feature, not a bug.
Unbiased (not family).
Diverse experiences.
Willing to provide raw, candid feedback.
You need to have the perfect idea in order to build something meaningful. - There’s no shortage of great ideas, just a shortage of people willing to put in the effort to capitalize on them.
You should say no to most opportunities. - As a general rule of thumb:
Early on, say yes - it puts you into growth-conducive situations. Saying yes allows you to expand your luck surface area.
Once you're established, say no - to focus and build leverage. Saying no allows you to focus on the 10x+ opportunities.
Free time is bad. - When you have free time, you have the headspace and bandwidth to pursue new ideas. Free time creates alpha.
Very few people have anything of value to offer. - Life gets so much better when you realize you can learn from anyone. No one is too old, young, rich, or poor to teach you something valuable.
Big changes will just happen on their own. - Nothing just happens - that’s for the movies, not real life. You have to kick down the door and blast through to the other side.
Your friends will always be there for you. - Your real friends are there when it is none of those things. They are there when you have nothing to offer in return.
You have to wait for luck to strike. - It’s hard to get lucky watching TV at home. It’s easy to get lucky when you’re engaging and learning - physically or digitally.
The world is a zero-sum game. - Celebrate everyone’s wins and you’ll start winning more. A rising tide lifts all boats.
Your GPA and test scores will define your future. - To be successful, you either need to learn how to build or you need to learn how to sell. People who know how to build AND sell are unbeatable.
The world is run by remarkable people. - Most of the people you admire are unremarkable.
Take all the advice that you can get. - Winners learn to filter and selectively implement advice - take the signal, skip the noise.
You have to be interesting to get ahead in life. - Interesting people sound impressive, but the reality is that being INTERESTED is more important than being INTERESTING.
You’re “too late". - This fallacy is driven by the fact that humans are terrible at comprehending exponential growth.
You can learn everything you need to know by reading books. - You can read every business and self-help book in the world, but ultimately the only way to learn is by f***ing it up.
There are shortcuts to achieve success in life. - The only “hack” is relentless consistency. It’s not glamorous and it's not fun but it works.
The smartest people always have the best answers. - The most intelligent people don't have the best answers - they ask the best questions.
You have to become an expert at something. - Many experts succeed because of the range of pursuits that preceded their main endeavor.
Most people are bad. - There are a few bad people out there, but most people are fundamentally good. They just want to be able to provide for their family and live well.
🎙️ The Vegan Dr 🌱
Dr. Neal Barnard was recently featured on an episode of the podcast, The Diary of a CEO by Steven Bartlett titled The Truth About The Vegan Diet, “Eggs Are Making You Fat”, “Stop Eating Cheese!”
🔑 Key Takeaways
Westernized diets, influenced by fast food chains, have led to increased rates of diseases like breast cancer, diabetes, and cardiovascular disease in countries like Japan and China. Dr Neal Barnard aims to promote plant-based diets to prevent these health issues.
By prioritizing fibre-rich foods and a plant-based diet, weight loss can be achieved without calorie restriction or intense exercise, leading to a boosted metabolism and improved digestion.
Increasing fibre intake through a plant-based diet rich in fibre can naturally aid weight loss by carrying unabsorbed calories out of the body, while also providing overall health benefits.
A plant-based diet has been proven to effectively treat diabetes, challenging the notion of it being a lifelong, incurable condition. Understanding insulin and glucose metabolism is vital in harnessing the benefits of this diet.
Adopting a plant-based diet low in oils and animal fat can reverse insulin resistance and diabetes by dissipating fat inside cells and allowing the insulin key to function properly.
Supplementation of vitamin B12 is necessary for those following a plant-based diet. Taking a B12 supplement is crucial, and following a predominantly plant-based diet has significant physical benefits. Transitioning gradually to a plant-based diet can lead to noticeable changes in weight, taste preferences, and overall well-being.
Adopting a plant-based diet not only improves health but also saves money. By focusing on plant-based foods and eliminating meat and cheese, individuals can save about $500 per year and reduce the need for medications and doctor visits.
Adding anthocyanin-rich foods like blueberries to your diet may lower abdominal fat, aid in weight loss, improve memory, and enhance cognitive function, especially in older adults. However, further research is needed for conclusive evidence.
Being aware of our biases is essential for making informed choices and evaluating information accurately.
The dairy industry's practices can have negative impacts on both animals and humans, including emotional distress for cows, weight gain and hormone imbalances for consumers, and contributing to the overconsumption of high-calorie cheese.
A balanced plate with a variety of plant-based foods and moderate protein is essential for optimal health.
Cruciferous vegetables like broccoli and cabbage are high in fibre and low in calories, making them effective for weight loss. They also have cancer-prevention properties and support overall health.
Our food choices play a significant role in our overall health. Opting for a diet rich in nutrient-rich foods and avoiding high-cholesterol foods like eggs can have a positive impact on weight loss and hormone levels.
Making dietary choices such as reducing fat intake, increasing fibre, and following a plant-based diet with low oil intake may lower estrogen levels, reduce the risk of breast cancer, alleviate menstrual cramps, improve menopausal symptoms, and optimize reproductive health.
Maintaining a healthy BMI and avoiding excess body fat, along with reducing dairy intake, can help optimize fertility by regulating estrogen levels and promoting a healthy reproductive system.
A properly functioning thyroid gland is essential for overall well-being. Iodine deficiency and antibodies attacking the thyroid gland are common causes of thyroid problems. A plant-based diet may help reverse hypothyroidism and reduce the risk of hyperthyroidism. Further research is needed for conclusive evidence.
Making healthier dietary choices, such as opting for blueberries over chips and avoiding saturated fats, can contribute to weight loss, reduce the risk of Alzheimer's, and protect cognitive abilities.
Prioritizing brain health through sleep, a healthy diet, exercise, and being mindful of medications can greatly impact memory and emotional control, regardless of age. Start now for long-term benefits.
Taking care of our health from a young age can prevent future health issues and promote overall well-being, including protecting against atherosclerosis and maintaining brain health.
Regular exercise, a healthy diet, and adequate sleep can reduce the risk of Alzheimer's disease. Including vegetables in your diet and being mindful of metal intake are also important factors in maintaining a healthy brain.
Being aware of the presence of aluminium in our daily lives and making conscious choices to minimize exposure can have a positive impact on our health.
By actively listening, conducting personal research, and focusing on progress rather than perfection, individuals can implement impactful ideas into their lives and create positive change.
All the above points have been taken from the podcast summary available at Podcast Disclosed. Please do listen to the podcast even if you are sceptical of the vegan diet.
❤️ Things I enjoyed
This section has been inspired by Ali Abdaal’s wonderful newsletter.
🎬 Video - THE A.I.//И.И. (Lazy Square & 1inch Collaboration III) is a funny animation about what would happen if we didn’t work with AI.
📝 Article - 20 of the coolest travel adventures for 2024 and Where to travel in 2024, without crowds highlight 2024 trends related to travelling.
📱 App - Download Microsoft Copilot on Android or iOS devices to start using GPT-4 without a paid subscription to OpenAI’s ChatGPT.
💬 Quote -
You get rich by owning things.
The biggest economic misunderstanding of my childhood was that people got rich from high salaries. Though there are some exceptions — entertainers for example — almost no one in the history of the Forbes list has gotten there with a salary.
You get truly rich by owning things that increase rapidly in value.
This can be a piece of a business, real estate, natural resource, intellectual property, or other similar things. But somehow or other, you need to own equity in something, instead of just selling your time. Time only scales linearly.
The best way to make things that increase rapidly in value is by making things people want at scale.
🎙️ Podcast - Harvard Professor: They’re Lying To You About Running, Breathing & Sitting! - Daniel Lieberman is an interesting episode from the podcast, The Diary of a CEO. It takes a completely different spin on our notion of exercise, the purpose of life, cancer, etc. The podcast summary is available here.
🎵 Song - Divya Drishti (दिव्य-दृष्टि) | DeeVoy Singh is an excellent song with some very intense lyrics.
🌐 Website - This web page is a simple printable calendar with the full year on a single page
Thank you so much once again for reading my newsletter this week. Please feel free to Buy me a coffee if you are enjoying what I am sharing.
If you want to follow me around the internet, then please do so here.
Until we meet again next week, walk more, listen more, be more interested and be more grateful for the things you have.
Join the conversation