Testing the user agent is set/ Postman/ proxy

Archive
1

Basics newbie question/stupid question .
Also, not HTTPS. I want to inspect the user agent string sent by a native app making a HTTP GET request. the native app runs on Windows/linux/macos , but for simplicity I want to run it on Windows. It makes HTTPS requests, but I’m forcing the native app to make HTTP requests to let me test with a local webserver I spin up on the same computer.

I thought that postman would be able to capture all traffic to port 8000 (I’m running a basic webserver on localhost.) So I installed the postman “Agent” and installed the postman “App”, so I’m not installing a browser extension at all, since the app under test is a native app and it does NOT have postman integration. What I’m trying to do should not work I assume? I could simply install wireshark, but that feels a bit OTT , or is it the only way to capture local traffic?

3 Likes
2

I’ve never tried using Postman as an intercepting proxy, so can’t help you there, but I’ve accomplished what you’re describing using Fiddler Classic as the proxy server.

2 Likes
3

Yeah - you can probably achieve it by running the traffic through a proxy such as Fiddler, Burp or Charles. Not sure how to set it up with a native Windows app, I have experience only with backend microservices and there it was super simple - when running the service locally, just set the proxy’s address in your environment variables.

Alternatively, you could check the user agent string in your backend logs, if you have access to those (edit: just realised you would have access to them since you’re running the server locally - so it’s just about whether the server is logging the agent string into stdout)

2 Likes
4

Yeah, because of various shortcuts forced by security concerns, the ability to run a true “integration environment” is infernally hard to set up, so I am dummying the “backend” which would normally log the user-agent strings for us and am running a simple java or python webserver on local machine instead. I could spin up a “fully decent” web server, but that would suddenly not be a one-liner anymore. I’m tempted to write a small web server that does log the string to stdout, good idea.

I have once used Charles, but found some sources of frustration and probably need to either use BurpSuite (which is an OTT tool for many simple things like this) or Charles I guess. I’m going to let you all know what I did do in the end, but am glad to know I was not struggling alone.

With each update of the native app, we modify the user agent string to include the version number, so this lets us track how many people are using old versions of the binary. So a very lightweight and in future easy to automate test is called for, and that’s why I’m going to have to share my solution later, which I’m pretty sure will be to use Burp.

1 Like
5

Ooooohkay. Now for the tricky part where I write a simple tutorial on how to test using Burp/Portswigger. I managed to get a good answer and this took putting two things in place, so first I’m going to be very clear about my goal again even if I did state it well in the beginning. And then… because I want to give you a full “lego-box”, I’m going to give you some easy to run code that will do the same thing.

The goal
I have an app that downloads a file from the internet over SSL, our web server tracks downloads by logging the agent string. I want to verify that string before each release, the agent string contains an expected version number from the downloading app, so I should be able to check it has the version number we expect, before we deploy. The app downloads a file over SSL on port 8000.

Advice
One “could” automate this, a few problems crop up though, because obviously the downloader uses SSL so it cannot be hijacked, the downloader uses a certificate chain to check. The downloader however has a way we can tell it to download using a URL we supply, as a test-hook. I also need to learn more about security testing tools, and my security expert has asked me to use Burp and become more familiar with the huge bag of other things the tool can do. This will NOT be using HTTPS even though it ideally should.

Steps
To keep things safe while we fiddle with proxy settings etc. I’m using a VM, Ubuntu is easiest.

  • Install the free version of Burp from here so that you don’t have to give them an email address.

  • Install python3. sudo apt-get install python3 Create a new folder this will be our dummy web server. Open it in terminal.

  • You can run a java web server, if java is your favourite tool but Python3 is an easier server for this. In that new folder in a console run python3 -m http.server 8080 . Or just use java

  • Create a blank text file in the folder call it README.txt

  • Open a browser on the VM an point to localhost:8080 , you will see your README.txt

    Screenshot 2021-12-08 133338
    Screenshot 2021-12-08 1333381037×650 60.4 KB

  • Now start Burp and go to the Proxy tab, change the port to 8000, and click on the actions tab and change the redirect so we redirect 8000 to 8080.

  • In Burp turn on the “interceptor”, its the blue button, if it’s not already on, and make sure Burp proxy does not say its “Not running”.

    Screenshot 2021-12-08 184536
    Screenshot 2021-12-08 184536891×490 55.5 KB

  • In browser go to localhost:8000 now and at the same time in Burp click the “History” tab to see the 2 requests the browser will make. Click on each one you will see a browser agent string, that’s the thing we want to test on the next step… time for our sample app… (In the next post)

6

Generally, if the application itself doesn’t have proxy settings, you need to proxy at the OS networking level. Can get a little hairy since all network traffic will then go over the proxy (as opposed to just the app of interest), but all of the intercepting proxies I’ve used have good filtering tools

3 Likes
7

Burp, part 2
(Actually since I’m not in the office, burp happens a lot, starting to like the swiss army knife look of this Burp tool.)

  • Copy this text into a file and save as download.py
#!/usr/bin/env python3
# Normally... downloads the file
# https://raw.githubusercontent.com/wp-cli/sample-plugin/master/readme.txt
# unless your run it with a parameter http://localhost:8000/
import sys
from urllib.parse import urljoin
import requests

headers = {
    'User-Agent': 'My User Agent 1.0'
}
url_file = 'readme.txt'
url_base = 'https://raw.githubusercontent.com/wp-cli/sample-plugin/master/'
try:
    url_base = sys.argv[1]
except:
    pass
url = urljoin(url_base, url_file)
response = requests.get(url, allow_redirects=True, headers=headers)
print(response)

You can choose to “forward” the request to our server by hitting the actions tab for the message, but we have what we need already.
That’s all there is to it, check that the headers and agent string is what we expect, job done. For fun, modify the program and run it again.