- Big Data News Weekly
- Posts
- 🤖 The Roadmap to Mastering LLM Inference Optimization
🤖 The Roadmap to Mastering LLM Inference Optimization
🦾Plus: 👕 ChatGPT can now virtually try on clothes for you
Sponsored by
Hey folks! Let’s get into Big Data and AI craziness…
In today's edition: Here is roundup of Olmo-core 3, Fourier Analysis, Analysis of variance👇
📊Analysis of variance--why it is more important than ever
🧠Ai2 released Olmo-core 3
⭕Fourier Analysis: Drawing Llamas with Circles
🤖Slovenia's .si Domain is Booming
💡 AI Tutorial:How to Get Started with Dots in ChatGPT
🤖AI Tools and Data Tools to checkout
In this article, you will learn how LLM inference optimization works and which techniques to apply to make language models faster, cheaper, and more reliable in production. How the two-phase inference process — prefill and decode — creates fundamentally different performance bottlenecks, and how understanding this distinction drives the choice of optimization technique.
State of Product reveals what AI still hasn’t solved
80% of product professionals say AI helps them ship faster, but customers aren’t seeing value any sooner. Atlassian’s State of Product 2027 explores the gains AI is delivering and the challenges that remain, from decision-making that hasn’t kept pace to gut instinct overriding customer evidence.
Data science is a rapidly evolving field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from structured and unstructured data. As the volume, variety, and velocity of data continue to grow, data scientists need sophisticated tools to handle and analyze this data
Analysis of variance (ANOVA) is an extremely important method in exploratory and confirmatory data analysis. Unfortunately, in complex problems (e.g., split-plot designs), it is not always easy to set up an appropriate ANOVA. We propose a hierarchical analysis that automatically gives the correct ANOVA comparisons even in complex scenarios…
🚀 Dive into the World of Machine Learning System Design! 🌐🔍
Ready to uncover the secrets behind how industry giants like Netflix, Airbnb, and Doordash leverage machine learning for unparalleled success? 🤖✨
📘 Download 'ML and LLM System Design: 800 Generative AI Case Studies & 45 ML Platforms': Unlock a treasure trove of insights with 800 real-world case studies from 100+ companies, delving into practical ML use cases and lessons learned in designing ML systems.
52,000 tokens per second per GPU against 19,400 in v2 on eight NVIDIA B300s, a 2.7x gain. The expert pool goes from 8 to 128 while holding about 3.2B active parameters per token, pushing total capacity from 4.6B to 47B, with a 1.2T-parameter run across 512 GPUs at 858 TFLOP/s per GPU. MXFP8 delivered 21% more throughput than BF16 and cut peak memory from 103 GiB to 95 GiB. Apache-2.0.
The Fourier transform is a method of transforming an input signal from the time domain to the frequency domain. It has a huge range of applications, for instance audio engineers can pick out individual undesired frequencies in a song with the Fourier transform and then get back the sound without that frequency using the inverse Fourier transform.
Leave Granola and get up to 12 months free of Wispr Flow Notetaker + Dictation
If you have paid time left on an individual Granola plan, we'll match it with a Wispr Flow subscription that includes Notetaker and dictation, and add bonus time, up to 12 months total. Sign in or create a Wispr account and submit proof of your plan to check eligibility.
👨💻 Data Tools, Libraries
Stirrup (GitHub Repo)
Stirrup is a framework for building agents that lets models choose their own approach to completing tasks.
ExecuTorch (GitHub Repo)
ExecuTorch is a solution for deploying AI models on-device. Built by PyTorch for privacy, performance, and portability, ExecuTorch powers KPWA meta's on-device AI across Instagram, WhatsApp, Quest 3, Ray-Ban Meta Smart Glasses, and more.
Actian VectorAI DB - Deploy AI agents with persistent memory. Ship them anywhere. Try Community Edition Free
AI News:
OpenAI is again experimenting with how its conversational AI assistant, ChatGPT, can help users as they shop online. On Thursday, the company announced the global launch of two new shopping features, including a way to virtually try on clothing and accessories and a new favoriting function that can help users save products they like for later reference.
200+ Proven Ways to Make Money With AI in 2026
The next wave of millionaires will be people who figured out how to make AI work for them.
The window to get ahead is still open. But not for long.
Here are 200+ proven ways to make money with AI in 2026.
Sign up for Superhuman AI, the free daily newsletter read by 1M+ professionals, and get instant access to all 200+ ways to profit from AI this year.
Slovenia just got handed the .ai boom's sequel, and nobody planned it. Registry SI told the BBC that .si registrations jumped 2,199% in September after an executive order required federal agencies to call AI super intelligence instead. Roughly 11,000 addresses landed on September 30 alone. Wix, for its part, reportedly sees no meaningful uptick yet.
SoftBank has officially wired its final $10 BILLION into OpenAI Masa had to sell his ENTIRE Nvidia stake to fund the initial payment and was so leveraged it had to issue an $11.1B junk bond to wire this final check
Turns out letting an AI agent read your entire hard drive has consequences, and Apple noticed. The company is adding new controls around macOS Full Disk Access, saying agents have increased the risks of that permission. It follows an Inc. columnist's claim that Meta's Muse app knew his private messages, which Meta disputes, plus a reported ChatGPT Mac flaw.
Boston Dynamics has given its Atlas robot a new pair of hands. But not just any hands. These hands drop the pinkies and only keep four fingers. The team decided to tape down their own pinkies to see what they'd miss.
AI Tutorial
How to Get Started with Dots in ChatGPT
Note: Dots is available to Pro and Business Premium users only, for now.
Open Dots in ChatGPT and follow the introduction. (You can create your dot only in the ChatGPT desktop app or in ChatGPT on desktop web.)
Connect apps such as email, calendar, and files so your dot can access their information and tools.
On mobile, open your dot's profile and go to Customize → Plugins.
You can also connect your computer from your dot's profile if you want your dot to work with local files, code, and apps.
After creating your dot, describe what you want it to do and provide the details it needs. You can ask it to review your calendar, research a topic, or remind you about something later.
Reply in the conversation if you need to add information or change the request. To attach a file or photo, select + in the conversation.
Dots start with built-in rules for when to act independently and when to ask for approval. On mobile, open Customize → Custom rules to review the defaults or add your own.
For each rule, choose whether your dot should act without asking, act if pre-approved, ask before acting, or hand off to you.
You can find out more about the new agents here.
Jobs:
This week's Data & Analytics jobs: I have updated my curated list with fresh openings. Browse and apply in one click: jobs
Machine Learning Engineer – Hybrid – Boston
Lead Machine Learning Engineer – Texas
Senior Data Engineer – Analytics – Boston
Data scientist - NY
🔥Top AI tools to increase productivity:
Agorapulse - LinkedIn's reach became erratic. Join Christopher Penn (Trust Insights) and Mike Allton (Agorapulse) live on October 1st
Archie: AI social content that sounds link you
Best Face Swap is an online tool that lets you replace faces in digital media using various AI models.
Framia Pro is an all-in-one AI platform for creating and editing images and videos
xLeadForge is an X outbound workspace for people who need to find and contact the right audience
Ito is an AI-powered quality assurance platform built for modern software teams
Creatii AI helps creators, marketers, and brands generate polished AI videos, images
dOCR is one API for documents and the web
View our database of all the best AI tools for your needs: aitoolsup.com
Have cool resources to share? Submit AI tool
A.I. Generated Image of the Day
👀 Great Wall of Lyrandia
Recommended reading:
SPONSOR US:
Get your product in front of Big Data & AI enthusiasts
😮 The Marketing Channel You've Probably Slept On:
Haven't tried newsletter sponsorships yet? You are missing out on a HUGE ROI+ customer acquisition channel. I know because dozens of advertisers keep coming back for more... Run a test campaign with us and see for yourself 👉 Get in touch today.
What did you think of today's email?Your feedback helps me create better emails for you! |