If you read part 1 of this post (and why wouldn’t do you that?) you’ll remember that I had just created a neural net using textgenrnn and Google Colab and then fed it 70,000 lines of breakfast cereal names and comic book titles. I wanted something like:

Captain America Berries – Whoops All ‘Merica!

Instead, what I got was alternating lines of comic book names and breakfast cereals. Never did the two mix. So, stuff like this:

Frosted Krispies (Kellogg’s)
The Best of Death and the Savage World Weird Worse Comics Stories
Sin City and the Call
Chocolatey Peanut Butter Crunch (Quaker Oats)
Quaker Cap'n Crunch Crunchberries
Corn Flakes (Erewhon)
Nick Fury's War

BOOOOORING!

So, after some dinking around I figured out what was going wrong. Long story short, the bot was too smart to be funny. Basically, it was considering too many nearby tokens (in my case, characters) to determine what to predict. This meant it was going to take a LONG time to make the funnies because there isn’t a lot of overlap here. I mean, words like “super” and “captain” would have got us there eventually I think. But I couldn’t wait for that. So, instead, I just made the bot stupider.

Here are the nerdy deets:

model_cfg = {
    'word_level': False,   
    'rnn_size': 128,  
    'rnn_layers': 3,   
    'rnn_bidirectional': True,   
    'max_length': 1,  
    'max_words': 100000,  
}

That max_length variable was set to 20 or 30. So, again, acting more as alchemist than scientist, I just cranked that sonofagun down to 1.

I then trained the model with this setting for 300+ epochs. The training configuration I was using would spit out some example text after every 10 epochs (NOTE: an epoch is basically one pass through your training data). I was starting to see some crossover between comic book and cereal names and, better yet, some of it was funny(-ish). It wasn’t all comedy gold, but there were definitely some nuggets in there.

With the model trained to this somewhat amusing level, I generated 3000 lines of output and curated it. I wanted to be very careful with the curation because I didn’t want the neural net to parrot my own ideas back at me. It’s way more fun to be surprised when the bot is hilarious. So, I decided that to do that somewhat impartially I needed ground rules.

Shane’s rules to nurture the AI

1. Cull the boring stuff: if the neural net gives me “Raisin Bran” I don’t need to reward that behavior. Raisin Bran isn’t funny.
2. Correct the typos: if the bot gives me “Batman: The Brave and the Otas” it’s ok to change Otas to Oats. It’s obvious what the funny robot meant.
3. Don’t over-correct the typos: if auto-correct would never suggest the word replacement that pops in my brain, I don’t get to use it, even if it’s hilarious.
4. Cull the garbage: if the text has more than 2 typos, I throw the whole line out. Even if it seems like it could be hilarious if I “creatively corrected” the typos.
5. Keep the random shit: sometimes I would get results that didn’t seem to be comic books or cereals. Things like “Don’t Be Petters“. I decided to reward the absurd, so this stuff stays in.
6. Keep funny typos: If there is a word that could be a funny cereal or superhero name, go ahead and keep it. For example “Almond Delight Wheat Sweeten Batties” just rolls off of the tongue and definitely feels like a superhero cereal I would enjoy. So, it stays.

I applied these rules to 3,000 lines out of output and whittled it down to roughly 300 lines. Here are some snippets:

Granola -How Fat with Raisins (Healthy Choice - Kellogg's)
Tony’s Tunch Cinnamon Cereal (Kellogg’s)
Bomb Queenity
Granola Low Fat wolf (General Mills)
All-American Spike
Avengers, Hearts Patrol
Bottle of the Face
Breade (Kellogg’s)
The Marvel Universe Comics Dick Files 2015
Cartoon Battle Shisters
Tomb Raisin Bran
Batman / Batman Occult Apples
Hawkman: Crunchy & Nick Flight

So, what’s next?

Well, obviously I fed those 300 moderately humorous lines back to the neural net and generated another 3000 lines output. Then I whittled that output down to roughly 1800 lines and fed it back to the bot. Based on the text I was seeing during the training output, it seemed like I was in a real good place. So, I generated 1000 lines of output and double-clicked the downloaded text file.

Then things got real funny.

The post Serial Adventures of Captain Cereal (part 2) appeared first on Chasing Tinfoil.

First thing’s first.

Before we get started on this adventure, we should probably talk about the chronology of this site. There is none.

I hope it isn’t too jarring but I’m just going to post projects whenever the heck I remember them, and certainly not in the order they occurred. Frankly, even the act remembering is a bit of a miracle, so just be happy that I post anything at all. However, all that said, this particular topic does happen to be the bright shiny thing I am currently chasing. So, what is it?

You guessed it. Time Machines! Wait.. no.. it’s Neural nets.

Why neural nets?

You see, since the early 1990’s I’ve been doing nerd work. And a big part of nerd work is log files. Loads and loads and loads of log files. So many log files that you definitely can’t read them all. But, if you don’t read any of the logs, you will eventually get caught off-guard.

To avoid this, I’ve been using regular expressions to filter my logs and e-mail the interesting bits to myself. I can’t just e-mail all of the logs to myself because, as I mentioned, there are way too many of them. Also, I don’t have the time or inclination to read all of the other e-mail I receive each day. Seriously. Stop sending me e-mail. Better yet, let’s BAN ALL E-MAIL!

No? Well, maybe some day. A guy can dream.

In the interim, and until we do live in this perfect e-mail-free world, I need to reduce the constant e-mail noise generated by servers, routers, firewalls, etc. To do this I use regular expressions to classify log entries. I have expressions for normal stuff (which is filtered out before I see it), bad stuff (which alerts me via e-mail) and worse stuff (which will send e-mail in ALL CAPS or something). Anything else that doesn’t match any of these expressions gets looked into at a later date to help me tune the expressions.

It’s not an ideal system but it’s better than nothing.
NOTE: If you are a system owner currently doing nothing, I hate you.

What can we use to fix this imperfect solution? You guessed it! Robots! Err… machine learning.

I mean, surely, if I train a neural net with buckets of good logs and bad logs, it can learn the difference and eventually be able to make decisions for me. Right?

Look, I just want a robot to do my job for me. Is that too much to ask?

No. No it’s not. I deserve a robot helper. I deserve one and no one is going to just give me one so it’s time to create one myself. But, how the hell do I do that?

How to create a neural net

After a bit panic googling, I found Amazon Sagemaker. This seemed like a good place to start. So I read some articles and did some labs and quickly decided that I was going to have to a) get a whole lot smarter really quickly or b) invest a whole lot of time that I didn’t have. This was discouraging.

But luckily the discouragement didn’t last long (this time I avoided the Swamp of Sorrows). Instead, my brain said, “Well, if it’s too much work to analyze text and make a decision, how about we just train a model to generate text. Super fun text! Something cool like… uh… comic books!”

Hey, yeah! Comic books are cool. I like comic books. Sure, let’s generate some fake comic book titles.

Well, It turns out that there are loads of ways to generate text with machine learning. I chose to use textgenrnn because I quickly found this pre-created Google Colab notebook that makes it super easy!

Now with comic books as my new focus, I pivoted and started training two models. I fed one of the models a list of every Marvel/DC comic book title. (roughly 35,000 titles) I fed the other model story names from those same 35,000 titles (which ended up being something like 300,000 total story names). The bots were fed and chock-full of comic book knowledge. Time to do damage.

NOTE: I got both title and story names from the Grand Comic Database, which is amazing and you should support it.

Once I got to the point where the models were reliably generating things that sounded like plausible comic book titles (“Batman: Ultimate Spider-Man”) and story names (“The Secret of the Sinister Secrets!”), option paralysis set in. Now what do I do?

Frankly, I’m still not sure. But hopefully I will come back to this and maybe create a twitter bot that will generate a comic book title name and story name, then do a google image search using that same text and finally generate a fake comic book cover image and tweet it out. We’ll see.

But I need something to do RIGHT NOW!

My brain must have sensed that I was dangerously close to the Swamp of Sorrows so it panicked and shouted “Cereal! Do something with breakfast cereal!”

NOTE: In this analogy my brain is a 5th grader who forgot he was scheduled to do a book report and is on the verge of tears.

Lucky for the troubled 5th grader (err… me, I guess), the distraction worked. It seemed like it would be hilarious to mash-up comic book titles and breakfast cereal names. I mean, I already had a bot that generated comic book titles. Creating another bot that did the same thing for breakfast cereals seemed trivial. After I had both, how hard could it possibly be to just mash them together somehow?

It was hard.

Look. I’m not doing science here, I’m doing alchemy. I barely understand the python neural net architectures I’m using. Scratch that. I don’t understand them at all. As such, I’m definitely not equipped to improvise on these architectures in code.

I am, however, equipped to fake it. So to the faking I did go! I grabbed the 600-ish cereal names and repeated them in a text file until I had roughly 35,000 total lines. Then I spliced this file with another file that contained 35,000 comic book titles.
NOTE: I used the Linux paste command for this

This gave me a 70,000 line file that was roughly a 50/50 mix of comic book titles and cereal names. Perfect food for a neural net that would, hopefully, produce hilarious mash-ups.

At this point I raced back to my Google Colab notebok and told it run through this file 100 times and learn from it. I assumed I could come back to find something hilarious like “Super Shredded Spider Bran“.

I assumed wrongly.

to be continued in Part 2

The post Serial Adventures of Captain Cereal (part 1) appeared first on Chasing Tinfoil.