- Big Data News Weekly
- Posts
- 🤖 The Complete Guide to LLM Evaluation
🤖 The Complete Guide to LLM Evaluation
🦾Plus: 💳 ChatGPT Can Now Shop and Check Out For You
In partnership with
Hey folks! Let’s get into Big Data and AI craziness…
In today's edition: Here is roundup of Google Orchestrator, ElevenLabs Notetaker, Claude meetings 👇
🔐Anthropic lets you run its agents behind your firewall
🎙️Liquid AI makes on-device voice agents faster
🧠How to train a transformer with no ML experience
⚡️ Google Launches Orchestrator
💡 AI Tutorial:How to Customize Spotify’s AI DJ Voices and Commentary
🤖AI Tools and Data Tools to checkout
In this article, you will learn how to evaluate LLM applications systematically, from choosing scorers and designing reliable LLM judges to evaluating RAG systems and agents across the release cycle. Topics we will cover include: How code, reference-based, and LLM-judge scorers differ, and how to write, score, and calibrate an LLM judge with rubrics, QAG, G-Eval, pairwise comparison, and Cohen’s kappa. How to score RAG systems and agents component by component, so you can tell which part failed.
Analytics on Live Data Without Leaving Postgres
When analytics on Postgres slows down, most teams add a second database. Then come the pipelines, the sync jobs, and a copy of your data that's always a little behind.
TimescaleDB takes a different approach: extend Postgres instead of splitting away from it. Hypertables partition your data automatically as volume grows. Hypercore compression cuts storage up to 95%. Continuous aggregates keep dashboards live without re-querying everything.
CERN runs Postgres this way for sensor data from the Large Hadron Collider.
No split architecture, no pipeline lag, no new query language to learn. Same SQL, same drivers, same tools.
Start on Tiger Cloud and get $1000 in credits.
Most software factory advice reads as a shopping list. Pick a platform here, a CI system there, add an AI reviewer, and the factory appears. It does not work that way, because these layers are not independent purchases. Each one is close to useless, and sometimes actively dangerous, without the layer beneath it
The AI lab just opened a private beta of self-hosted Claude Managed Agents, so your team can run the entire agent stack in its own cloud or on-premise. That includes sandboxes, credentials and session history, making it easier to put agents to work fixing builds and triaging alerts on sensitive code. Anthropic is also rolling out Claude Meetings in beta, which joins your video calls, creates docs and whiteboards as you talk, and saves notes afterward.
🚀 Dive into the World of Machine Learning System Design! 🌐🔍
Ready to uncover the secrets behind how industry giants like Netflix, Airbnb, and Doordash leverage machine learning for unparalleled success? 🤖✨
📘 Download 'ML and LLM System Design: 800 Generative AI Case Studies & 45 ML Platforms': Unlock a treasure trove of insights with 800 real-world case studies from 100+ companies, delving into practical ML use cases and lessons learned in designing ML systems.
The AI startup just shipped LiquidCodec, a ~74M-parameter model that turns speech into tokens language models can work with, then converts them back into audio. Liquid AI says it encodes speech 361x faster than real time on a single CPU thread, making it roughly 13x faster than Kyutai’s Mimi. That could mean quicker responses and smoother streaming for your voice agents, even on edge devices. It runs through Hugging Face or llama.cpp.
This guide follows a software engineer building and training a transformer from scratch for a real coding challenge. You’ll learn how to turn application state into tokens, use distillation and self-play, shrink the model to fit tight runtime limits, and test whether better training scores actually translate into better results in the real system.
22 ChatGPT Agents Built for Every Marketing Job
Most marketers use ChatGPT to do general research and then call it an AI strategy. The ones outperforming them are deploying specialized agents built for specific jobs.
We put together 22 plug-and-play ChatGPT marketing agents that handle the work eating your week, each with built-in instructions and structured outputs ready to go in under 5 minutes.
Subscribe to Marketing Against the Grain and get all 22 free.
Inside you'll find:
Competitive intelligence agent that visits competitor websites and builds detailed comparison matrices automatically
Customer feedback analyzer that ranks improvement opportunities by business impact
Social listening specialist that monitors brand mentions and flags reputation risks before they escalate
Campaign optimization agents that handle attribution analysis and surface what is actually driving results
Your competitors are already running agents like these.
Get 22 ChatGPT Marketing Agents free when you subscribe to Marketing Against the Grain today.
👨💻 Data Tools, Libraries
Get ready for the Claude Certified Architect exam with Packt's FREE 45-minute webinar! Explore exam topics, watch a live AI agent demo, practice exam questions, and grab your FREE Starter Kit!
Try Snowflake free for 30 days with $400 in credits and run your analytics, pipelines and AI on one platform. Load your data, run SQL at any scale, and build pipelines and ML models without managing infrastructure.
VectorAI DB: Do you need a vector database? Find out in 30 seconds. Before you budget for a vector database, check whether you need one. Answer a few questions about your data, latency, filtering and deployment.
AI News:
ChatGPT just asked for your credit card, and this time it's not a scam. OpenAI launched ChatGPT Wallet, which lets its AI agents (called dots) pick out products and complete the checkout on your behalf. It's starting with a limited rollout to U.S. Pro subscribers with a Visa, Mastercard or Amex card, you approve each purchase with Face ID or a fingerprint, and yes, you still keep your card rewards.
Your buyer asked ChatGPT. It named your competitors.
Search didn't die. It moved.
Your buyers ask ChatGPT, Gemini, Claude and Perplexity what to buy, and act on the answer. If you're not in it, a competitor is.
ChatFeatured does the work to get you cited. Most customers see their first citations inside 7 days.
Google rolling out Gemini 4 Hydrogen inside Google Antigravity alongside a new model dropdown option called Orchestrator, which pairs Hydrogen with our strongest models for long-horizon tasks, including Gemini 4 Carbon, in an optimized balance of quality, efficiency, and latency.
Move over Granola, ElevenLabs wants a seat in your meetings too. The $22 billion voice AI giant released ElevenAI, an experimental app that combines meeting notes, dictation in over 90 languages and AI agents that can handle follow-ups like drafting emails. It's still in preview behind an early-access signup, and it's pitched at teams that would rather not pay for separate notetaker licenses.
Claude can now join your video calls as its own participant and work on tasks while you talk. Anthropic’s own example is a call about planning an event. Halfway through, you ask Claude to start the plan, and it drafts one from what everyone has said so far. When the budget comes up, it asks your permission to look up what last year’s event cost and lines that up against the new plan.
A day after Notion's AI chats leaked in the Fireworks AI hack, Uber says it got hit too... Uber is now notifying thousands of drivers that their personal and driver's license info was stolen when hackers broke into the vendor in October. Filings show 2,567 affected in Massachusetts, 3,430 in South Carolina and 1,873 in Rhode Island, and that tally is expected to rise.
AI Tutorial
How to Customize Spotify’s AI DJ Voices and Commentary
Open Spotify on your mobile device and use the search bar to look for DJ.
Tap the three-dot menu to access the voice and commentary settings.
Browse the nine new voice options and select the one you prefer:
Casey [US]: Laidback energy, warm, bit of a drawl.
Tess [US]: Direct but inviting, has a bit of an edge. In-the-know.
Lane [US]: Southern accent, brings a younger, positive energy.
Sam [AU]: Gregarious, fun, likes to keep things not-so-serious.
Nell [UK]: Mellow, slower energy, and deeply knowledgeable.
Declan [IR]: Straightforward, passionate, and a bit of a music nerd.
Nisha [IN]: Inviting, with contagious energy.
Nico [MX]: Down to earth, present, and right there with you.
Fabi [MX]: Witty, friendly, and always a step ahead.
If you prefer listening without frequent interruptions, you can now turn off commentary between songs. Your AI DJ will only speak to introduce the listening session or when it's directed to change up the music choices.
Jobs:
This week's Data & Analytics jobs: I have updated my curated list with fresh openings. Browse and apply in one click: jobs
Machine Learning Engineer – Hybrid – Boston
Lead Machine Learning Engineer – Texas
🔥Top AI tools to increase productivity:
Marblism generates a fully-functional web application from a single prompt:
Clipwing A tool for cutting long videos into dozens of short clips
aiPDF is an innovative, multi-modal tool designed to work with a wide array of inputs, including ebooks, web articles, YouTube videos, podcasts.
xLeadForge is an X outbound workspace for people who need to find and contact the right audience
Ito is an AI-powered quality assurance platform built for modern software teams
Creatii AI helps creators, marketers, and brands generate polished AI videos, images
dOCR is one API for documents and the web
View our database of all the best AI tools for your needs: aitoolsup.com
Have cool resources to share? Submit AI tool
A.I. Generated Image of the Day
👀 Autumn Sci-Fi
Recommended reading:
SPONSOR US:
Get your product in front of Big Data & AI enthusiasts
😮 The Marketing Channel You've Probably Slept On:
Haven't tried newsletter sponsorships yet? You are missing out on a HUGE ROI+ customer acquisition channel. I know because dozens of advertisers keep coming back for more... Run a test campaign with us and see for yourself 👉 Get in touch today.
What did you think of today's email?Your feedback helps me create better emails for you! |