Search notes:
python standard library: urllib.parse
parse_qs / urlparse
from urllib.parse import parse_qs
from urllib.parse import urlparse
url = 'https://server.tld/path/to/resource;par?value-one=foo&value-two=bar&value-two=baz#xyz'
parsed = urlparse(url)
print(f'scheme: {parsed.scheme }') # https
print(f'netloc: {parsed.netloc }') # server.tld
print(f'path: {parsed.path }') # /path/to/resource
print(f'params: {parsed.params }') # par
print(f'query: {parsed.query }') # value-one=foo&value-two=bar&value-two=baz
print(f'fragment: {parsed.fragment}') # xyz
print('')
values = parse_qs(parsed.query)
print(values['value-one']) # ['foo']
print(values['value-two']) # ['bar', 'baz']
print(values.get('unobtainium', ['n/a'])[0]) # n/a
Note: the values in the
dict returned by
parse_qs are
lists!
quote /quote_plus
Note the different treatment of the space and the forward slash in quote and quote_plus:
import urllib.parse
print(urllib.parse.quote( 'AAA BBB/CCC')) # -> AAA%20BBB/CCC
print(urllib.parse.quote_plus('AAA BBB/CCC')) # -> AAA+BBB%2FCCC
Non ASCII characters:
print(urllib.parse.quote( 'ä ö ü')) # -> %C3%A4%20%C3%B6%20%C3%BC
print(urllib.parse.quote_plus('ä ö ü')) # -> %C3%A4+%C3%B6+%C3%BC
unquote / unquote_plus
unquote decodes
percent encoding.
unquote_plus additionally replaces a
+ with a space:
import urllib.parse
print(urllib.parse.unquote('x%2fy+z')) # x/y+z
print(urllib.parse.unquote_plus('x%2fy+z')) # x/y z
urlencode
urlencode (presumably) uses
quote to create a
URL query from the key values given in a
dict:
from urllib.parse import urlencode
urlencode({'num': 42, 'txt': '"hello" <world>'}) # --> num=42&txt=%22hello%22+%3Cworld%3E
urljoin
import urllib.parse
print(urllib.parse.urljoin('https://tld.xy/path/to/xyz.html', '/abc.html' )) # https://tld.xy/abc.html
print(urllib.parse.urljoin('https://tld.xy/path/to/xyz.html', 'abc.html' )) # https://tld.xy/path/to/abc.html
print(urllib.parse.urljoin('https://tld.xy/path/to/xyz.html', '../abc.html' )) # https://tld.xy/path/abc.html
print(urllib.parse.urljoin('https://tld.xy/path/to/xyz.html', 'https://another.tld.xy/foo/bar/baz.hml')) # https://another.tld.xy/foo/bar/baz.hml
print(urllib.parse.urljoin('one/two/three.html' , 'four/five.html' )) # one/two/four/five.html