Recent searches
Search options
For those of you who have a high-volume input channel and, recently, software offering to summarize it with AI… Do you summarize much/at all?
[Maybe boost for reach, really interested in the result here.]
@timbray I'd be super interested to see how this breaks down by age.
@timbray LLMs are entirely the wrong tech for summarization. LLMs give you "average text". useful summaries tell you what diverges from average.
@timbray I used to think it would be a reasonable task to allow LLMs to have a go at, but reading this article put me off that idea:
https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/
@timbray not sure if this is in the vein of your ask, but I’ve been pumping my “read laters” into Listen Later, which generates audio and puts it in a podcast feed for me. It’s really increased my actual intake of things I wanted to “read”.
@timbray folks that understand how AI works triage before accepting the AI answer. Sadly far too few folks understand how AI works.
@timbray Our block has a high-volume WhatsApp with lots of back and forth about free things being offered, requested favors, jokes, etc. I get a lot of value from iOS summarizing 12 messages mid-workday to “Avocado procured; free kids’ books offered”
@timbray For now I skip past the AI summary, and usually the sponsored results. I wish I could find the article with the URL parameter to add to your search to make it like "ye Google of olde"
@timbray Although I answered the poll, on reflection I suspect I don't actually have a high-volume input channel, and my "never!" vote might not actually be helpful.
If we posit that you're not going to be able to consider 100% of your input, you have to triage and discard some portion of it.
Triage procedures are not intended to be perfect (because if they were, we would have fully considered all the input, which is the stated problem and not it's own solution!)
Given that triage isn't perfect, does performing triage on AI summaries give a worse result than triaging the original articles?
We have two main sets of input documents; those that are easy to triage (because they already have easy-to-read summaries & conclusions), and the rest. Using an AI to summarise the first set would probably be bad; but using it on the second set would probably be an improvement the process, making triage faster.
Yes, there's still the case that the AI summary is just plain wrong, but you were only using it on the hardest-to-triage articles in the first place, and if their authors didn't care enough about them to write actual summaries, it's probably no great loss to you.
@timbray Oh, miss interpreted the question. So this is about things like high traffic soccer mom whatsapp groups and the like. I use LLMs to summarize long emails, which have been going back and forth between three parties for weeks, before it reaches my desk. Completly different ballpark.