Reading Instagram Private API Requests with Charles Proxy
To automate the management of Instagram accounts, I use various libraries that use the so-called Private API.
Libraries generate requests, pretending to be the official application, but over time Instagram changes its API and the libraries lose their relevance.
I have to wait for the community to fix the problem, or fix it myself.
Initially, the idea seems sound, the community around the library will help in its development, but in fact everyone is waiting for someone to solve the problem that has appeared.
To take control of the generation of requests, you can write your own library, for this you need to constantly disassemble the official application, decompile it and study the logic of work.
This is a difficult way, the code is poorly readable at the output, and logic is also smeared throughout the application, you have to restore the event logic bit by bit to generate a request.
There is an easier solution - sniffing traffic from the application to the API, as a result, any request will be immediately visible and there is no need to collect it piece by piece.
But there is a problem - all requests to the Private API go through HTTPS. It is impossible to read such traffic, it is encrypted.
Solution: In the Facebook settings (for Instagram), you can enable user certificates and disable TLS, that's what we'll do in this article!
We will use Charles Proxy to solve the problem. With its help, we will monitor traffic from an Android smartphone to a desktop (in my case, this is macOS Mojave).
Install Charles Proxy, it will launch a proxy on your desktop and offer to install its certificates (Help > SSL Proxying > Install Charles Root Certificate on Mobile Device or Remote Browser):
Add Charles to the macOS firewall (Settings > Security & Protection > Firewall):
Disable caching in Charles (Tools > No Caching):
Now we need to go to Researcher Settings on Facebook and enable custom certificates for your Instagram account:
Check the boxes next to "Enable user installed Certificate Authorities (CAs) for your Facebook account" and "Enable user installed CAs for your Whitehat Test Accounts." And select Instagram under "Select on which apps you want to enable the Mobile Settings".
Stop the Instagram app on your smartphone and clear the data for this app. Launch the Instagram app again and click "Login with Facebook". Then go to Settings > System Tools > Whitehat Settings and enable "Allow user installed certificates" and "Do not use TLS 1.3":
Install Charles certificate on smartphone
Remember, your smartphone and desktop must be on the same local network!
In the WiFi connection settings, specify the proxy whose address was obtained in the very first screenshot of Charles:
As soon as requests go through this proxy, Charles will offer to confirm the connection, click Allow:
Open a browser on your smartphone and follow the link chls.pro/ssl, you will be prompted to save the certificate. Open the certificate via "Install to certificate":
And install it:
Now let's make any request through the Instagram app (just reload feed) and mark the hosts to use SSL:
As a result, you should display a tree of requests to the Private API:
Well, the requests themselves with headers will also be visible:
However, if your traffic looks something like this:
So you misconfigured user certificates, leave a detailed comment on this article and I'll try to help you!
But if you can read the headers and body of the request still in binary form, then see the Accept-Encoding header. If zstd is specified in it, then the request body can be read using my utility zstdcat.
pip install zstdcat cat /tmp/response.bin | zstdcat