Open in app

Sign In

Write

Sign In

Less Wright
Less Wright

1.91K Followers

Home

About

May 14

What LayerNorm really does for Attention in Transformers

2 things, not 1… Normalization via LayerNorm has been part and parcel of the Transformer architecture for some time. If you asked most AI practitioners why we have LayerNorm, the generic answer would be that we use LayerNorm to normalize the activations on the forward pass and gradients on the backward. But that default…

Artificial Intelligence

3 min read

What LayerNorm really does for Attention in Transformers
What LayerNorm really does for Attention in Transformers
Artificial Intelligence

3 min read


Jul 11, 2021

Sculpted shoulders — the best exercise for the rear delts

Building from the V shaped physique concept, you need to also have depth from the side view. This is achieved by building the ‘rear delts’, and here’s the most efficient exercise for quickly doing that. The rear delts are often ignored in many workout routines — and they are hard…

Body Image

3 min read

Sculpted shoulders — the best exercise for the rear delts
Sculpted shoulders — the best exercise for the rear delts
Body Image

3 min read


Jul 5, 2021

Achieving V shaped physique — the optimal shoulder exercise to add shoulder width

Exercising your side delts can add impressive width, but one exercise gives you the best bang for the buck. Based on studies, women are most attracted to men with a V shaped physique, meaning the shoulders are wider than the waist, creating a V taper from the front view. Given…

Bodybuilding

3 min read

Achieving V shaped physique — the optimal shoulder exercise to add shoulder width
Achieving V shaped physique — the optimal shoulder exercise to add shoulder width
Bodybuilding

3 min read


Jun 4, 2021

A working, evidence based theory on the origins of Covid-19: hint, mice with human lung tissue, molecular dynamics, lab leak.

Genetically modified mice with ‘humanized lungs’ were apparently created to test the infectiousness of coronaviruses in Summer, 2019 and not Spring 2020 as portrayed. Unlike other zoonotic jumps ala SARS, Covid-19 was surprisingly ‘pre-adapted’ to thrive and transmit in the human respiratory system from the start. Molecular modeling shows Covid-19…

Coronavirus

9 min read

A working, evidence based theory on the origins of Covid-19: hint, mice with human lung tissue…
A working, evidence based theory on the origins of Covid-19: hint, mice with human lung tissue…
Coronavirus

9 min read


Jun 1, 2021

Rapid fat loss, keep the muscle— has the optimal combo of diet and exercise been found?

Combining intermittent fasting days with cardio = Double the fat loss, retain muscle mass, improve cholesterol… A study that compared the effects of intermittent fasting by itself, cardio by itself, or the combination of both may have found the optimal setup for rapid fat loss whilst maintaining muscle mass (and…

Diet

5 min read

Rapid fat loss, keep the muscle— has the optimal combo of diet and exercise been found?
Rapid fat loss, keep the muscle— has the optimal combo of diet and exercise been found?
Diet

5 min read


Jan 4, 2021

Building Bigger Biceps — 4 science based tips that can maximize your results.

Four tips from scientific studies point the way to maximizing your gains if you are interested in increasing your arm size as efficiently as possible! 1 — Train your arms first. While conventional workouts typically follow the larger muscles first, then smaller muscles, multiple studies have shown that muscles worked…

Workout

4 min read

Building Bigger Biceps — 4 science based tips that can maximize your results.
Building Bigger Biceps — 4 science based tips that can maximize your results.
Workout

4 min read


Dec 27, 2020

Summary info on how the new mRNA Covid-19 vaccines (Pfizer, Moderna) work— no DNA reprogramming, no microchips:

Short summary of how the US approved mRNA Covid vaccines work (Pfizer, Moderna) With my work in Covid-19 related AI, I frequently get asked about the various covid vaccines, both approved and pending. Thus, I’m writing this article to provide a useful summary of the current mRNA vaccines, and will…

Covid-19

5 min read

Summary info on how the new mRNA Covid-19 vaccines (Pfizer, Moderna) work— no DNA reprogramming…
Summary info on how the new mRNA Covid-19 vaccines (Pfizer, Moderna) work— no DNA reprogramming…
Covid-19

5 min read


Jan 5, 2020

Meet AdaMod: a new deep learning optimizer with memory

AdaMod is a new deep learning optimizer that builds on Adam, but provides an automatic warmup heuristic and long term learning rate buffering. …

Machine Learning

5 min read

Meet AdaMod: a new deep learning optimizer with memory
Meet AdaMod: a new deep learning optimizer with memory
Machine Learning

5 min read


Dec 26, 2019

Meet DiffGrad: New Deep Learning Optimizer that solves Adam’s ‘overshoot’ issue

DiffGrad, a new optimizer introduced in the paper “diffGrad: An optimizer for CNN’s” by Dubey, et al, builds on the proven Adam optimizer by developing an adaptive ‘friction clamp’ and monitoring the local change in gradients in order to automatically lock in optimal parameter values that Adam can skip over.

Machine Learning

5 min read

Meet DiffGrad: New Deep Learning Optimizer that solves Adam’s ‘overshoot’ issue
Meet DiffGrad: New Deep Learning Optimizer that solves Adam’s ‘overshoot’ issue
Machine Learning

5 min read


Dec 16, 2019

Reducing your labeled data requirements (2–5x) for Deep Learning: Deep Mind’s new “Contrastive Predictive Coding 2.0”

Current Deep Learning for vision, audio, etc. requires vast amounts of human labeled data, with many examples of each category, to properly train a classifier to acceptable accuracy. By contrast, humans only need to see a few examples of a class to begin properly and accurately recognizing and classifying future…

Machine Learning

5 min read

Reducing your labeled data requirements (2–5x) for Deep Learning: Google Brain’s new “Contrastive
Reducing your labeled data requirements (2–5x) for Deep Learning: Google Brain’s new “Contrastive
Machine Learning

5 min read

Less Wright

Less Wright

1.91K Followers

PyTorch, Deep Learning, Object detection, Stock Index investing and long term compounding.

Following
  • The Serenity Research

    The Serenity Research

  • Erik Engheim

    Erik Engheim

  • Brokkr Finance

    Brokkr Finance

  • Trademaster NTU

    Trademaster NTU

  • Ariel Protocol

    Ariel Protocol

See all (17)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech

Teams