Rather than making a webpage and upload images by the user, I found that directly apply style transfer to Instagram images will be more fun. So I tried to combine the knowledge I learned from Hacking The Browser to make a chrome extension.
For this project, I think it can be separated into three parts:
The first part is train my own style base model. Originally I wanted to follow this repo to train my own model. However, it takes too long for me and I cannot make my own style model, so I can only take three trained model from this repo.
The second part is to do style transfer. Thank to wonderful ml5, it gets the jobs done easily. However, I don’t know if the style transfer function in ml5 has a little problem on getting image width and length. Because I found that the if the image is not a rectangle image, the output image will become a mess and also the width and the length of the output image seems to be swapped.
The third part is to make a chrome extension. Since I am going to get the image data from Instagram, there is a cross-origin data issue due to network security, so I have to find other approaches to get the image data.
I tried to download the image I chose so that the image I am going to deal with will become a local data. At first I thought the extension will pack all images onto chrome so the web accessible resources (images) will also unchangeable. Luckily it seems that it will still change if I download the image to my extension folder.
So downloading the image is somehow a little bit weird, so I tried another way after the presentation. I tried to get the screenshot of the page and cut the part I want (which is the image I clicked). So this time I send the image to background to do style transfer. At first I thought doing style transfer in background won’t slow down the user doing things at front. However, it seems that it will slow down the whole browser. Fortunately, the image is not that large, so it only takes few seconds. However, when you hover to the image, there will come up with a div, and I cannot get rid of it, so the output image will have the div on it.
The full code is here: StyleTransfer