Blog

Hacking The Browser HW3

For the first part, I put in github: Bookmaklet2Extension

For the second part, I chose a bookmarklet called “MapThis”, which can grab the address you select and directly redirect to google map. The code is showing below with comments on it.

 

For the third part, I chose an interesting chrome extension called Giphy Tabs, which will come up with a GIF every time you open a new tab.

NOC Final Project Proposal

For the final project, I am interested in working on machine learning concepts so I want to take this chance to make a machine learning / neural network project.

In Jabrils’ talk, he showed an image style transfer project:
https://github.com/luanfujun/deep-photo-styletransfer

Since I was working on some image processing projects and I am really interested in combining machine learning techniques into it.

Also, I found that there are some projects like Face Portraits Style Transfer , Style Transfer and Colorizing Grayscale.

So I want to take images and transfer them into a style of different kinds of paintings.

I don’t  know if it is too ambitious to do this or even makes sense to do this since there are already some developed projects or even mobile apps that can do these works.

RWET HW6: Semantic Similarity

For the homework this week, I choose to implement word semantic vectors on the poems I found from Poetry Foundation

So I went to the page of Cat Poem Collection
https://www.poetryfoundation.org/collections/101638/cat-poems
to get all the poems in the page. Then I randomly chose 10 sentences from all these poems to be my poem base.

I followed the tutorial from Replacements and walks with word vectors to do word vectors. I use spaCy’s word vectors to search similar words of my base, then give 50% of the chance to change the word.

Hacking The Browser HW2

To make a bookmarklet that makes user difficult to use, I try to put in the event of “mouse not moving” feature that I asked last week.

So the bookmarklet has 2 parts, one is when the mouse moving and another one is when the mouse is not moving. When the mouse is moving, I take popular tags (p, li, h1~h6, span) and grab their innerText, then the text will keep disappearing in each event trigger. Moreover, the user will find that there are still some links that seems can escape from this weird webpage by clicking it. When the link is clicked, it will redirect the page to google search with the keyword “antivirus”.

As to the not moving part, when the mouse is still for more than 2 seconds, the opacity of the entire page will keep decrease, and the page will finally become totally white.

the code is here: hw2

the codepen is here: Codepen for HW2

NOC Week 9: Neural Network

For the exercise this week, I simply want to do some practice on making a neural network. So I choose to follow the playlist of Doodle Classifier.

I grabbed 4 datasets (mosquitos, airplanes, birds, and dragons) from Google Quickdraw, which I want to see how would the performance be if  my target data have similar characteristics (they all have wings). Instead of using Processing to parse and reduce the data, I use Python to read the npy file and save them as txt file.

At first, I even wanted to write the neural network by myself. Unfortunately, I was stuck with a problem when I was making the testing function. With 4 outputs, the % of correctness should be around 25% without training. However, every time I run the testing function, the % of correctness is always exactly 25%, it is even the same 25% when I add the training process, which is so weird. (And strangely at the time I am writing this blog, it works…)

So I tried different number of nodes to see the correctness in 10 epochs.

It seems that adding nodes doesn’t improve the correctness, I am wondering if the training data is not enough or it is simply because these data are too similar.

here is the code: hw6

Hacking the Browser HW1

To make a “responsive” page, I tried to interact with the browser window itself. Since I have no experience with jQuery and some experience with javascript, I am still trying to get used to it in this homework.

So I got some inspirations from the “browser pong” page. I made a button to popup a small page and whenever your mouse cursor is approaching the page, it will try to move away from your cursor. If you can hit the button in the popup page, then you simply win!

At first I got a straightforward thought, I want to detect the mouse position when the mouse cursor is out of the window, but I found that it seems that it is not allowed. So I changed to detect the mouse movement on the window, then move the position of the window. Moreover, I want to come up a text when the popup window is closed, and I use “focus” event listener to do it.

Also, I am wondering if there is an event listener for the mouse is NOT moving? So that I can do something when the event is not occurring. And is it possible to callback a function from a different page?
(So the problems are solved, THANKS!)

The code is here: hw1

NOC Week8: Genetic Algorithm

To finish the homework this week, I got an idea from the smart rocket example. It makes me think of a kind of shooting game – bullet hell.

It is a shooting game, however, there are a lot of bullets coming out so you have to dodge them.

So I tried to make a genetic algorithm to train the rocket to dodge the bullets.

For the output of this GA, it will be the net force to move the rocket, and I set it to detect the distance between the rocket and each bullet. If the distance is shorter, the force magnitude should be larger.

mag(force) = X * 1/distance

So “X” is what I am going to put in rocket’s DNA.

For the rewarding (fitness) mechanics, it simply get higher points with longer living time.

I make 50 rockets (populations), so one generation will finish when all the rockets hit a bullet.

 

However, it seems that it needs a LONG time to make it live longer…
or sometimes I found that sticking in the corner will live longer…

genetic algorithm

Detourning the Web HW4

For the homework this week, I make a mail bot to send mails about some tweets in specific hashtag on Twitter.

For the Twitter page, it needs to turn off javascript to get the content in every tweet. Also, I found that the page will be redirected to the mobile version.

For the keyword, I found #TeachMeSomethingIn5Words was on a trend and it seems pretty interesting so I use this hashtag as my target.

 

So I also packed those tweets with images and attached them in the email. However I found that there are a lot of tweets with images are politic-related…

 

Back to the code, since it downloads everything by using beautifulsoup so the data I get is literally a list of words and a list of images. I have to know which tweet need an image. So I found that when you attach an image on Twitter, there will be an url for the image, so I use regular expression to search for the pattern of “pic.twitter.com” to distinguish which tweet contains image.

Here is the code: hw5

RWET HW5: Markov Chain

For the homework this time, I simply play around with Markov chain and ngrams on the data I scraped from Yahoo Answer.
I get the questions on  the page and make them as a string/list, then I make different Markov chain model with characters and words.

With ngram = 2, model level = char:
the output will be kind of nonsense

With ngram = 5, model level = char:
the words of the output are more reasonable

With ngram = 2, model level = word:
most of the words are reasonable, and the output text is longer than the previous one

With ngram = 5, model level = word:
most of the outputs are concatenations of the original questions

When ngram increases, it makes sense that there are few options for a pattern.
However, since the dataset I got is pretty random, it is easy to see some of the outputs following the original question sentence.
Also, if I put the questions in a list and I only use markov_model() function to generate the model, it is easier to get EOT.

Detourning the Web HW3: 4 Panel Comic

For the homework this time, I tried to grab a video from Youtube and turn them into images and make a series of 4 panel comics.

The source video is from Jimmy Kimmel Live and I found that the interview is a good material to be a 4 panel comic.

I use youtube-dl to download the video and the subtitle. Then I use videogrep to search the interview part by using “who, why, who’s” as the keywords. After getting the supercut, I use ffmpeg to turn the video into images with fps=1 so that I won’t get tons of images in my folder.

Since I don’t need every image made by ffmpeg, and at first I wanted to use the subtitle to get the specific frame. However, the timestamp of the subtitle is not precise, which will come up with the frame that I don’t want. Also, sentences of the subtitle will repeatedly come out due to the frames it calculated. Accordingly, I can only manually choose the images and the subtitle I want.

This can be an example of the repetition of the subtitle sentences

 

As to grabbing the sentences in my supercut, I found that when running videogrep, it will print out the sentences it is using when making the supercut. So I go to the source code file and find the function that prints out the message and add few lines to output them into a txt file.

 

Now I have images and sentences I want, it is time to use Pillow to put them together.

 

Although it contains some manual parts, I still put them into one python code file: hw3