Yahoo Groups (Fandom) Rescue Project

While we can use any language skill (there are tabs in over a hundred different languages!), we have a particular need for help with Farsi. Google Translate expects it to be written in its version of the Arabic script, but much of the Farsi on Yahoo Groups was written in the Latin alphabet, which Google Translate won't process for that language. (My last attempt was guessed as "Bengali" despite being clearly Farsi, and was naturally rendered as almost complete gibberish.) This means we absolutely need a native speaker to read the text and tell us what's being said.

Are you able to do this? Do you know someone who can? Please pass this request on and let them know to check out our Discord server to volunteer their help.

One of the Unknown groups currently being worked on is in Farsi and we're stuck without help from someone who can read it.

EDIT: I have been able to make use of this converter but it's an extra step that slows everything down. Native speakers are more accurate, plus they recognize and understand context details that can make all the difference with tagging.

Tagging has slowed somewhat, as people have gotten busy with school and work. There are still over 5000 tabs to tag. However, experienced taggers have tagged nearly 50 apiece. If we had 100 taggers tackle the project right now and stick with it the way some of our volunteers have, it would be finished very shortly!

So this is once more a call to those of you who are interested in seeing this complete: If you want to help, consider either tagging a tab, advertising/recruiting others to tag (see the boost text from this post if you need it), or both!

I spent several years working on the metadata at the expense of working on projects that would help support me financially, so I've had to step back from actively working on it for now. However, if all the tabs get tagged, I will absolutely be there to sort the actual data. I'd love to see it organized and uploaded!


Actual stats:

Now up to 4.02% tagged.

Available tabs (sorted by descending numbers by language):

English: 3208
Unknown*: 544
Spanish: 387
Portuguese: 336
French: 146
Indonesian/Malay: 131
Italian: 84
German: 77
Turkish: 64
Chinese: 59
Arabic: 54
Romanian: 36
Spam**: 32
Persian: 16
Dutch: 12
Filipino: 12
Swedish: 8
Hungarian: 7
Polish: 7
Vietnamese: 6
Bosnian: 3
Finnish: 3
Catalan: 2
Danish: 2
Esperanto: 2
Lithuanian: 2
Norwegian: 2
Russian: 2

Single tabs available:
African: Afrikaans, Chichewa, Hausa, Kinyarwanda, Malagasy, Somali, Swahili, Yoruba
Asian: Acehnese, Armenian, Azerbaijani, Batak Toba, Bengali, Georgian, Gujarati, Hebrew, Hindi/Urdu, Javanese, Kannada, Kapampangan, Kazakh, Korean, Kurdish, Malayalam, Marathi, Mongolian, Sundanese, Tamil, Telugu, Tetum, Thai, Turkmen, Uzbek, Uyghur
European: Albanian, Basque, Croatian, Czech, Estonian, Galician, Greek, Icelandic, Ido, Interlingua, Latin, Latvian, Maltese, Occitan, Slovak, Slovenian, Welsh

All compilation tabs are still available as well.

* As stated in a previous post, the "Unknown" groups are by and large not unknown as far as language goes, but they can't be tagged without looking at the actual messages. Most are clearly in English.

** As stated in a previous post, the spam groups will also need to be looked at a little to confirm spam status, but many are in clear patterns and most of the groups won't need to be looked at once one or two are. I have no idea what language, if any, most of those are. My suspicion is that they were created just for email address harvesting, but we may never know for sure.


Something fun:

I've been using Random.org to generate id numbers for groups in the metadata database to give those in the Discord server a randomized peek at the variety of groups we knew about (and mostly saved). Here's one of the fan groups that came up in that process:

BetteMidlerADivineGathering
created 2004-10-23
/Entertainment & Arts/Celebrities/
Looking for a place to stop and talk about the Divine Miss M? Here's your place! Feel free to join, post messages, pictures, files, or even chat! EVERYONE is welcome!
Just remember...keep this place clean and free of hate and negativity. Bashing of Bette Midler or members of this group will NOT be tolerated. Of course everyone is entitled to their own opinions, but if you have something negative to say to another group member, please keep it away from the group...email them personally.
And remember to HAVE FUN!!!
Thanks,
Dusty
61 members

The tagging process is moving out of beta mode; while minor improvements may be made over time, we are hopeful that no big changes will need to be made from here on.

We therefore invite any and all willing volunteers to come help. Attention to detail, the ability to work in Google Sheets/Docs, and the willingness to ask questions when in doubt are all you need.

What does tagging involve?

Tagging is the process of reading through group metadata (names, descriptions, and a couple other useful fields) and deciding what the group is about. Then you copy/paste the correct information into specific columns. You will be assigned a 'tab' of groups to keep the process manageable. Each tab has between 100 and 300 groups on it, from one or more categories on Yahoo. You can work at your own pace and drop in when you have time.

The guidelines for the process are in a Google Doc and cover most of the situations you'll encounter. If you run across something difficult, we encourage you to paste it in the correct channel on our Discord server and ask for advice.

Can I ask to tag something specific?

Yes, but you're limited to a) categories that Yahoo actually had, and b) things that haven't been tagged yet. Not all fandoms that were popular during that time had their own categories, and except for the very large (such as Harry Potter, Star Trek, Dragon Ball, or Backstreet Boys), most will be on a tab with other fandoms. (When you volunteer to claim a tab, you're volunteering to tag all of the groups on that tab, not only the categories you're interested in.) Nonfandom areas also vary wildly in their size; there are enormous quantities of classmate and alumni groups, software groups, genealogy groups, romance groups, recycling groups, and adoption groups for instance, but there may only be one or two groups for a particular automotive make or health condition.

Tagging isn't exceptionally difficult, however, and once you successfully complete one tab, you will probably find the second one even easier. Plus, you never know what gems you may run across while you do it!

What if I'm not comfortable tagging groups? Is there any other way to help?

Yes, there are many other ways to help.

1. Importing
We need people who are willing to install an email program called Sylpheed on their computers to import Yahoo Groups into a format that will help us tag. Not everyone can install the program, so you can help with importing. That way we have a steady flow of groups for people to tag.

2. Languages
If you can read a language other than English, you can help on the server by opting into a role for your language. Then if someone is trying to tag groups in that language, they can call on you for help with anything confusing.

3. Boost!
We also need more visibility on this project, which brings us to…

Can I advertise this somewhere I know?

Yes, please! There are about a million groups left to tag, on over 5000 tabs. The more volunteers we have, the fewer tabs each person has to tag.

If it's useful, feel free to share this text in quotes (change the first word to whatever number is accurate, since time marches on - the project started in 2019):

Five years ago, with little notice, Yahoo announced Yahoo Groups was being deleted. An army of archivists swung into action and saved nearly a million groups in all - 14 terabytes of data.

The next step of the Save Yahoo Groups project is tagging the groups.

We need volunteers who:

* are detail-oriented and careful

* ask lots of questions when in doubt

* can use Google Sheets/Docs at a basic level

Also helpful:

* able to read languages other than English

* able to install a simple program and follow a visual guide to importing mbox files

* extensive knowledge about a particular subject

If this interests you, check out our Dreamwidth community and volunteer to help: https://yahoogroups.dreamwidth.org/profile

I finished all the German categories and have been going through Portuguese ones. Oddly enough, unlike the other categories, the Portuguese ones started out somewhat jumbled, though it seems to have stabilized mostly going through top-level categories one at a time (but in reverse order alphabetically, chunk by chunk). After the Portuguese, the only language directory that's left is French, which I suspect will be relatively smaller overall.

I did find a new language—Cape Verdean Creole—I think! Google Translate thought it was Brazilian Portuguese but it's definitely not and all research that volunteers did for me seems to turn up Cape Verdean Creole, so I'm assuming that must be it. Hopefully we can find someone who can read it in order to tag it properly.

This will be, I believe, the final update for metadata sorting until I'm finished. If my calculations are correct, the 95% mark will occur in the middle of the final category, which is the massive NULL one (groups for which we have no information on category path, category, or categoryid). Sorting this category will go remarkably fast, because there are only three possibilities: we have at least a partial description which can be used to tag (likely to be very rare), we have no info but we have the GMD so the mboxes can be looked at (also probably rare), or we have no info at all (the most likely). The first can stay on the appropriate language tab, the second and third go to Unknown. After that, I have only the final 1-2% to handle - finalizing all the smaller language tabs (including some that turned out to have far more groups than I imagined and will require splitting into multiple tabs). I can't wait to be done with this stage!


Actual stats:

Now up to 90.00% sorted and 2.89% tagged.

Available tabs:

English: 3254
Spanish: 388
Portuguese: 269
Italian: 88
German: 76
French: 8
Chinese: 59
Indonesian/Malay: 130
Arabic: 54
Persian: 16
Turkish: 63
Romanian: 35
Unknown: 281
Spam: 31


Something fun:

Yahoo Groups were used to host hundreds of groups containing downloads of custom content for The Sims. Such groups could be found in many languages, as the group "the_sims_downloads" shows:

Comunidade destinada à downloads do jogo The Sims 1,2,3 e no futuro 4.
Este grupo aqui no yahoo, é para fazer downloads dos jogos da série The Sims(TS1,TS2 e no futuro TS3).
Aqui terão vários tipos de downloads e também dicas sobre os jogos da série The Sims.

I'm still sorting jumbled categories, often sets of related ones, such as sports by location, or individual family groups by letter. I've had sets of dog breeds, soccer/football categories, and schools in various countries to sort, along with scattered TV shows and movies, and lots of actors and music artists/bands. There have also been quite a few role playing categories, as well as categories specifically for fanfic for various fandoms.

I found a couple new natural languages—Luganda and Ladino—as well as some constructed languages.


Actual stats:

Now up to 80.01% sorted and 2.81% tagged.

Available tabs:

English: 3112
Spanish: 349
Portuguese: 13
Italian: 88
German: 5
French: 8
Chinese: 52
Indonesian/Malay: 127
Arabic: 52
Persian: 15
Turkish: 61
Romanian: 35
Unknown: 252
Spam: 31


Something fun:

Anyone into history or costuming might find "HistoricCostuming_EdwardianWW1" interesting (it was one out of a whole series, with groups for every era imaginable):

This group is for the discussion of, sharing research about and reconstruction of Historic Costume/Fashion during the Edwardian period through World War 1 (1900-1920AD). This is a public group that any serious person is welcome to take part in. I encourage using the resources available on the Yahoo Groups site to share information.

This has still been a big jumble of English categories - everything from military groups to dog breed groups to various actors, music artists, and authors (of which they had a set by letter, like the previous sets for actors, music artists, movies, and TV shows). I also hit the Chinese categories, which are slower to sort (detangling the Unknown from groups that are in English and Chinese both from groups that are only in Chinese but use English for fandom titles). Since I can't easily tell what they are unless I copy/paste them into Google Translate, any Chinese speakers offering to tag (and I hope there are some eventually!) will definitely need to look up which cat_ids they want, so I can identify the correct tab. Fortunately there weren't many Chinese groups initially—mostly in fandom categories, especially computer & video games—so I got through them and back to the jumble of mostly English categories with the occasional Spanish one. They do pop up now and then later on, but in smaller and more manageable batches.

Found a few new languages - Venetian, Xhosa, Lojban, Monda, Aranese, and Coptic!


Actual stats:

Now up to 70.00% sorted and 2.67% tagged.

Available tabs:

English: 2692
Spanish: 296
Portuguese: 11
Italian: 88
German: 5
French: 7
Chinese: 44
Indonesian/Malay: 114
Arabic: 47
Persian: 14
Turkish: 53
Romanian: 33
Unknown: 208
Spam: 30


Something fun:

Some group descriptions just make me chuckle, like that of "Cat_Trek":

These are the voyages of the starship Catstongue. Their mission: to boldly meow where no cat has meowed hitherto. Come join the crew as we explore the deepest darkest furtherest reaches of space and time in our neverending quest for the lost planet of Catalonia. You will be assigned a Trek identity, and you can contribute to the continuing adventures being recorded in the Captain's Log. OR you can just join to have a good read! The latest cast list can be found in the shared files, under the cast list folder! New members might like to look here before they choose a character for themselves.

Among the many varied categories this time was a whole series of categories for music artists in general, by letter. A had lots of Aaliyah and Avril Lavigne, for instance, while M had My Chemical Romance and Michael Jackson among many, many others. There was also a similar series of categories for actors, and for computer & video games, though the last set was much smaller overall.

I also finally hit the Spanish categories, as you can tell by how the number of available Spanish tabs shot up suddenly. These are by far easier to sort than English tabs, for although I have to be careful to spot the occasional Catalan and Basque groups mixed in (and once in a blue moon, Galician), there's little Unknown (I can't skim the Spanish as easily as I do English, and I'm less certain of whether someone will be able to tag them or not based on what's there, and have chosen to err more on assuming they can), no spam so far, and virtually none of the languages so prevalent in the English categories (Indonesian, Arabic, Turkish, Persian, etc., very few other languages at all). The Spanish fandom categories are almost entirely Spanish with a very rare Catalan group; I mostly see the odd other languages in nonfandom categories.

I did find a new language - Mayan!


Actual stats:

Now up to 60.01% sorted and 2.67% tagged.

Available tabs:

English: 2390
Spanish: 126
Portuguese: 10
Italian: 87
German: 4
French: 6
Chinese: 8
Indonesian/Malay: 102
Arabic: 43
Persian: 13
Turkish: 49
Romanian: 31
Unknown: 184
Spam: 30


Something fun:

Sometimes the interesting part about a group was who created it or was part of it, like with "heroesiiimapmakers":

Greetings. This club is for Heroes of Might and Magic III, IV and V fan's that like making their own maps. Here you can get and exchange tips and tricks on how to make a great map for the game. I am a former Level Designer for New World Computing's Heroes of Might and Magic II and III series. Though I no longer do it professionally but I still love making maps as a hobby. This club is not affiliated with New World Computing, The 3DO Company or Ubisoft in any way. This is strictly group for fans.
Now that Heroes V is here let's meet here often and share our initial impressions of the new game.

Halfway done with sorting! I hope the second half goes quicker than the first.

These categories have been quite a wild jumble of mostly smaller categories that Yahoo created later. More Cultures & Community, more Regional, more Health & Wellness, more Music, more Entertainment & Arts, more Computers & Internet, more Science… and yes, more Schools & Education and Romance & Relationships. I've even run across a new language: Georgian!

As I sort, I'm reminded afresh of some of the reasons why this project is important. True, some groups, like the ones filling the Technical Support category (for long-obsolete devices and operating systems), are relics of another time and are largely useless to modern users. But others - such as the ones sharing tubes for Paint Shop Pro (which, as far as I can tell, can still use the old files) - may still be relevant to people today. Not to mention the sheer amount of creative work of all types that is preserved only in the messages and files of various groups.


Actual stats:

Now up to 50.01% sorted and 2.45% tagged.

Available tabs:

English: 2001
Spanish: 29
Portuguese: 16
Italian: 87
German: 3
French: 5
Chinese: 8
Indonesian/Malay: 88
Arabic: 39
Persian: 12
Turkish: 43
Romanian: 29
Unknown: 155
Spam: 26


Something fun:

Someone will surely find the group "lostcities" interesting:

This group explores the legends and reports of lost cities, lost continents, lost communities and lost peoples around the world, and highlights the real-life expeditions that have set out to find them.

Did Plato's Atlantis exist, and if so where was it? What happened to British explorer Percy Fawcett, who vanished in the Amazon while searching for a lost city? Is it time for a revival of "lost race" novelists like Canada's James DeMille ("A Strange Manuscript Found in a Copper Cylinder" - 1888) and America's William Starbuck Mayo ("Kaloolah, Or Journeyings to the Djebel Kumri" - 1849)?

The illustration at right is Maxfield Parrish's "City of Brass" (1909), a bookplate rendition of the Saharan lost city depicted in the Thousand and One Nights.

Facebook group: http://www.facebook.com/group.php?gid=247655422288

Well, I hit some of the real tedium. Schools & Education - including two of the five largest categories - generally for class groups, whether it's the "everyone taking this section of Chemistry" or the "everyone graduating from this school in this year" type. They weren't very interesting to sort and will probably be less interesting to actually tag, but they have to be gone through. The smaller categories specifically for educators will be more interesting, I suspect, but the classmates/alumni sorts are really, really not - and there are a LOT of them. That, plus real life projects and events, is why this update has been later than usual.

I didn't find any new languages, but I did find a group in Armenian which used the Armenian alphabet. That was interesting! The other two Armenian groups I'd seen didn't, and I haven't seen Armenian enough to recognize it without the alphabet, but with it, I was able to ID it quickly; the alphabet is quite distinctive.


Actual stats:

Now up to 40.04% sorted and 1.57% tagged.

Available tabs:

English: 1614
Spanish: 24
Portuguese: 13
Italian: 86
German: 3
French: 4
Chinese: 7
Indonesian/Malay: 67
Arabic: 28
Persian: 10
Turkish: 34
Romanian: 25
Unknown: 108
Spam: 20


Something fun:

While most of the classmates groups were quite boring, there was the occasional oddity, like the group called "school-wedgies":

Do you get wedgies at school? Do you give wedgies at school?
If you do this is the group for you!

I've been sorting through games, games, and more games. Particularly notable was the sheer number of groups for Sims CC and Freedom Force meshes and skins. (I was one of the ones who specifically hunted for Sims CC groups so that at least does not surprise me one bit.) And there were an enormous number of RPGs, more than I ever knew existed before. I have literally dozens of tabs of purely RPGs awaiting tagging.

After the games were a host of categories for law and lawyers, military, and politics. Then a set of categories for health-related groups—doctors, support groups for medical conditions, fitness & weight-related, pregnancy-related, etc. I've just finished those and have begun sorting groups for hobbies and collecting.


Actual stats:

Now up to 30.01% sorted and 1.40% tagged.

Available tabs:

English: 1169
Spanish: 18
Portuguese: 9
Italian: 85
German: 2
French: 3
Chinese: 7
Indonesian/Malay: 38
Arabic: 22
Persian: 8
Turkish: 22
Romanian: 9
Unknown: 93
Spam: 17


Something fun:

The group "allyourbasearebelongtous" made me chuckle, with description below:

In A.D. 2101
War was beginning.
Captain: What happen ?
Operator: Somebody set up us the bomb
Operator: We get signal
Captain: What !
Operator: Main screen turn on
Captain: It's You !!
Cats: How are you gentlemen !!
Cats: All your base are belong to us
Cats: You are on the way to destruction
Captain: What you say !!
Cats: You have no chance to survive make your time
Cats: HA HA HA HA ....
Cats: Take off every 'zig'
Captain: You know what you doing
Captain: Move 'zig'
Captain: For great justice
Put related pictures in the Photos section.

Fortunately, we were able to save this group's Photos section. (This was only true of ~5% of the groups we saved.)

(There was also a group devoted to the Zero Wing game, titled "someonesetupusthebomb".)