March 26, 2020
API
Sniffing
Android
Instagram
macOS
Proxy
Debug
По-русски
Reading Instagram Private API Requests with Charles Proxy
To automate the management of Instagram accounts, I use various libraries that use the so-called Private API.
Libraries generate requests, pretending to be the official application, but over time Instagram changes its API and the libraries lose their relevance.
I have to wait for the community to fix the problem, or fix it myself.
Initially, the idea seems sound, the community around the library will help in its development, but in fact everyone is waiting for someone to solve the problem that has appeared.
Reverse Engineering
To take control of the generation of requests, you can write your own library, for this you need to constantly disassemble the official application, decompile it and study the logic of work.
This is a difficult way, the code is poorly readable at the output, and logic is also smeared throughout the application, you have to restore the event logic bit by bit to generate a request.
There is an easier solution - sniffing traffic from the application to the API, as a result, any request will be immediately visible and there is no need to collect it piece by piece.
Proxying requests
But there is a problem - all requests to the Private API go through HTTPS. It is impossible to read such traffic, it is encrypted.
Solution: In the Facebook settings (for Instagram), you can enable user certificates and disable TLS, that's what we'll do in this article!
We will use Charles Proxy to solve the problem. With its help, we will monitor traffic from an Android smartphone to a desktop (in my case, this is macOS Mojave).
Install Charles Proxy, it will launch a proxy on your desktop and offer to install its certificates (Help > SSL Proxying > Install Charles Root Certificate on Mobile Device or Remote Browser):
Add Charles to the macOS firewall (Settings > Security & Protection > Firewall):
Disable caching in Charles (Tools > No Caching):
Now we need to go to Researcher Settings on Facebook and enable custom certificates for your Instagram account:
Check the boxes next to "Enable user installed Certificate Authorities (CAs) for your Facebook account" and "Enable user installed CAs for your Whitehat Test Accounts." And select Instagram under "Select on which apps you want to enable the Mobile Settings".
Stop the Instagram app on your smartphone and clear the data for this app. Launch the Instagram app again and click "Login with Facebook". Then go to Settings > System Tools > Whitehat Settings and enable "Allow user installed certificates" and "Do not use TLS 1.3":
Install Charles certificate on smartphone
Remember, your smartphone and desktop must be on the same local network!
In the WiFi connection settings, specify the proxy whose address was obtained in the very first screenshot of Charles:
As soon as requests go through this proxy, Charles will offer to confirm the connection, click Allow:
Open a browser on your smartphone and follow the link chls.pro/ssl, you will be prompted to save the certificate. Open the certificate via "Install to certificate":
And install it:
Now let's make any request through the Instagram app (just reload feed) and mark the hosts to use SSL:
Resume
As a result, you should display a tree of requests to the Private API:
Well, the requests themselves with headers will also be visible:
However, if your traffic looks something like this:
So you misconfigured user certificates, leave a detailed comment on this article and I'll try to help you!
But if you can read the headers and body of the request still in binary form, then see the Accept-Encoding header. If zstd is specified in it, then the request body can be read using my utility zstdcat.
pip install zstdcat cat /tmp/response.bin | zstdcat
Comments
Hi...thanks you for the explanation...I usually reverse engineer apps with mitmproxy..so tried it with Instagram after attempting to remove the ssl pinning (I didn't know there was a researcher settings at the time) needless to say...it wasn't a successful operation.
Now, after reading your explanation I attempted to read the requests with charles and enabled the research settings on Instagram...the requests however are marked as 'unkown' and as if it was sent from a normal user account with no testing enabled...I've eliminated the possibility that I could have misconfigured the CA certificate as I can read the requests sent via the mobile browser.
So I'm not sure what is causing this problem...your help would be greatly appreciated.
Thanks.
This is the error I'm getting in charles: "SSL handshake with client failed: CA certificate could not be matched with a known, trusted CA (unknown_ca)"
"You may need to configure your browser or application to trust the Charles Root Certificate. See SSL Proxying in the Help menu."
I'm using Android, not IOS in this case.
Reply here
Post your comment