u/8ta4

I'm thinking about building a tool to discover backronyms for initialisms, like "Married But Available" for MBA. Since the potential search space for these word combinations follows V^n, where V is the vocabulary size, finding funny sequences is a challenge.

I've mapped out a workflow:

1. Seeding. Extract over 10,000 English initialisms from Wiktionary.

2. Filtering. Use a recognizability dataset to reduce the list to a subset that most people would know.

3. Mining. Match these seeds against the Google Ngram dataset for 2- to 5-gram sequences.

4. Ranking. Categorize the resulting phrases by their initialism and sort them by frequency, capping the count per bucket to keep the volume manageable.

5. Judging. Use a large language model as a judge to scan the lists for funny expansions.

My biggest concern with this approach is the frequency distribution. "Married But Available" does appear in the Google Ngram dataset. But it's roughly a million times rarer than a sequence like "May Be A". If the funny candidates are buried too deep in the tail, they might be dropped before the model sees them.

Does any systematic solution or dataset for this problem already exist? Any other feedback is welcome.

I'm looking for a way to filter buffers in-place while keeping the data editable. It's like an Excel spreadsheet: you filter the rows to hide the noise and continue to work on the data within the sheet itself.

Telescope doesn't work for this. I want to navigate and edit within the filtered results in the main window.

Here are the features I'm looking for:

• Regex search: The input must support regular expressions.

• Conceal-based masking: Non-matching lines are hidden using extmarks and the conceal attribute.

• Async interruption logic: Typing triggers a background regex scan. The logic is async so the editor doesn't freeze while processing 1M+ lines. Too big? That's what Sheet said. If I type a character before the previous scan finishes, the old job must be killed.

• 1-row input bar: The UI is a 1-row floating bar that pops up when a filter is active. It stays open while I swap focus between the bar and the buffer. If I clear the filter, the bar goes away.

Does anyone know of an existing plugin that handles this type of masking?

I checked out the word prevalence dataset of 62,000 lemmas. But it has some limitations:

• It hasn't been updated since 2019.

• It misses modern terms like TikTok.

• It doesn't cover phrases.

I've scored about a million English entries from Wiktionary for recognizability. I built this for a pun tool. But I want to use the data for a new language project.

The dataset is too bloated because it's full of inflected forms. Even if I set the recognizability threshold at 50 percent, I'm still looking at 100K words and 100K phrases. Going through a list that size is a waste of time. I need to filter the data through the English lemmas category from Wiktionary and split the single words from the multi-word phrases into separate lists.

Since the hard part of scoring is done, the rest should be easy peasy lemma squeezy. I just want to avoid reinventing the wheel if I can.

Before I spin up a separate repository to handle this, I'm checking if a similar dataset already exists. Has anyone seen a project that offers this?