https://nwilliams030.substack.com/p/10-frontier-tech-the-great-month?utm_source=email

Thanks for holding as I paused last week, and made a small personal announcement :)

nicole ruiz @nwilliams030we’re having a baby! a Ruizlet.

September 20th 20227 Retweets873 Likes

This week I decided halfway through editing that I needed another special edition entirely devoted to ML - large language models (LLMs), image generation, and general tooling progress as it’s shot forward at a blistering pace. If this is up your alley, scroll on, if not - will be back to the regular cadence next week.

ML Research Developments

• 8.22.22 → We started out with the public release of the stable diffusion model by the Stability AI folks (almost exactly a month ago … wild).
• 9.9.22 → New state of the art (SOTA) open CLIP models to drive image classification and generation forward (”Large Scale Open Clip” from the folks at LAION, supported by Stability AI). CLIP models are embedded into DALLE2, Stable Diffusion, and Midjourney’s generative models.
• 9.7.22 → On the auto-code production side, Github ran a study that showed a 50% decrease in time to write a web server in JavaScript when using GitHub Copilot (which turns natural language prompts into code suggestions).

Nat Friedman @natfriedman👀

September 8th 2022199 Retweets2,544 Likes

• 9.9.22 → Runway dropped perhaps one of the sickest beta teasers** of all time, demo-ing text to video features that span importing background footage, removing background objects, generating background images, green-screening, and more that will be deployed soon in beta. (**We are investors in Runway).

Runway @runwaymlMake any idea real. Just write it. Text to video, coming soon to Runway. Sign up for early access:

runwayml.com

September 9th 20222,994 Retweets15,869 Likes

• 9.14.22 → Adept dropped their ACT-1 Transformer for Actions which they trained on a number of complex software tools such as Salesforce or Excel that typically also require complex navigation ability. Adept’s query box beautifully allows natural language voice or text to command complex tasks in Salesforce, Excel, etc and also takes feedback. Adept @AdeptAILabs2/7 This can be especially powerful for manual tasks and complex tools — in this example, what might ordinarily take 10+ clicks in Salesforce can be now done with just a sentence. September 14th 202230 Retweets316 Likes

  

• 9.11.22 → Run stable diffusion locally on your mac with Diffusion Bee.

• 9.17.22 → Six days later, this guy also created a way for you to run stable diffusion locally on your mac. No internet, rate limiting, or account necessary.

• 9.19.22 → Interesting use case of both stable diffusion and CLIP: reverse image search. Lexica.art enables you to input any photo and come up with stable diffusion generated similar photos, using clip-ranking/retrieval.

Sharif Shameem @sharifshameemJust added reverse image search to Lexica. You can upload a photo and it’ll return the most similar Stable Diffusion images and their prompts. This makes it very to turn a real world concept into a Stable Diffusion prompt.

lexica.art

September 19th 2022262 Retweets1,663 Likes

• (As the creator notes, this is particularly helpful for finding images where knowing the appropriate natural language query / successful CLIP prompt is hard. Think: you want to generate a specific texture or an abstract organic shape. Image retrieval, and also general image sorting in latent space is increasingly important but hard to do swiftly [One example might be that you want a group of images that all are similarly inspired, but hard to describe - ‘clothing where the texture is inspired by or visually similar to grass,moss, or dirt’ would be a possible set of images that you might want to prompt.] I’m certainly excited the applications that come out of this.)
• 9.21.22 → Open AI’s dropped Whisper, an open-sourced neural net that achieves human level robustness and accuracy on English speech across accents and technical language and it can run locally! Descript watch out!