Repository navigation
Fails validation on URLs that are not FQDNs #217
Description
A URL without a FQDN is perfectly valid. The resolver may attempt to add labels to a non FQDN in order to fully qualify it allowing the user to specify non-FQDNs.
I.e. http://pc:8081/ is perfectly valid:
$ host pc
pc.example.com has address 10.75.22.1
$ curl -q -4 -v 'http://pc:8081/'
* Trying 10.75.22.1:8081...
* Connected to pc (10.75.22.1) port 8081 (#0)
> GET / HTTP/1.1
> Host: pc:8081
> User-Agent: curl/7.82.0
> Accept: */*
>
Because /etc/reslov.conf has search example.com in it.
Yet this fails validation with validators.url()
$ python -c 'import validators; print(validators.url("http://pc:8081/"))'
ValidationFailure(func=url, args={'value': 'http://pc:8081/', 'public': False})
Activity
miigotu commentedon Aug 29, 2022
I don't think this is a bug in validators. It was just a hasty expectation that it would act similar to urlparse, but validators is using a complex set of checks and regexes that can't tell what's in your resolve.conf or hosts file or is set up as a docker endpoint for example. The fix is just for me to know how it works and use it correctly.
SickChill/sickchill@65a4c34
brianjmurrell commentedon Aug 30, 2022
I disagree.
It shouldn't need to. It just needs to accept that unqualified host/domain names are perfectly valid and as such don't HAVE to have multiple labels in them and validate them as valid.
While it seems that that would work-around the issue in validators by preempting it with
urlparse, I don't think it changes the fact that a single label host/domainname is valid. It may or may not actually yield a resolvable IP address, but as a name, it's still valid, just like a FQDN may not resolve an IP address but validators will validate it as valid.miigotu commentedon Aug 30, 2022
Ok, but you always disagree. validators works just fine. Maybe you should write a replacement 🙄
brianjmurrell commentedon Aug 30, 2022
Yes, I will raise an issue when an implementation has a bug in it -- hopefully to prevent others from getting bitten by it. But I guess you think that when a bug is found I should just be quiet about it and/or instead rewrite the whole project? That makes sense.
miigotu commentedon Aug 30, 2022
It isn't a bug though. When a piece of software doesn't work how YOU want it to, but it is working exactly as the developer intended, it is not a bug.
validators.url is designed specifically for FQDN, using regexes, not checking if it is a named endpoint network link, or otherwise.
A bug would be, if the dev intended it to be a certain way, documented it a certain way, and it doesn't work that way. You're idea of what something SHOULD be doesn't determine what it was intended to be.
What you are trying to say, with really bad people skills, is that you wish it had a feature that made it work how you think it should work.
The library is meant to help weed out bad data, not to fit 100% of edge cases. If it allowed unqualified names to match without disabled-by-default parameterization it would make the library practically useless, because most bad urls that are mistakenly entered or make it somewhere look exactly like local net endpoints or docker endpoints.
Now, I fixed MY mistake in the code in SC. None of that anywhere meant that validators was broken.
If you always think you're the smartest person in the room, you usually aren't.
brianjmurrell commentedon Aug 31, 2022
It is though.
Documentation (and therefore developer) says:
url
validators.url.url(value, public=False)[source]
Return whether or not given value is a valid URL.
A non FQDN is a valid URL. Chrome opens it.
curlopens it. Any valid URL resolving tool will open it if the resolver is configured accordingly.Is it? The documentation for it doesn't even mention FQDN. Even if it did, it would not be a valid URL validator. It would be somebody's non-inclusive idea of what they think a URL is which would only include a subset of actual valid URLs.
Again, the documentation (and therefor dev.) says:
Return whether or not given value is a valid URL. And that is it. There are no caveats about FQDNs. A non-FQDN is a valid URL. Nothing in the documentation says it needs to be FQDN. So according to the documentation, the dev intends for it to be a valid URL even if it's not FQDN. If the dev. intends to only include the subset of valid URLs that doesn't include FQDNs, the documentation should list that limitation.
A non-FQDN is hardly an edge-case, IMHO.
Validation is hard.
I guess we will just have to agree to disagree.
This project and it's maintainer are of course completely free to disagree with my bug report and close the ticket and do nothing about it. That is fine. My effort here was just to prevent future users of
validators.url()to fall into the erroneous handling situation that I found myself in when using a non-FQDN URL. So go ahead and close this issue if you still feel it is invalid and I won't say anything more.miigotu commentedon Aug 31, 2022
A named endpoint is not a URL, Have fun here though, because you won't be irritating me on SC anymore..
By RFC 1738, the host in a URL is REQUIRED to be an IP or a FQDN, not an unqualified domain name. Not a local hostname. Not a netbios name, not a network resource link. Modern browsers just happen to ALSO work with resource links, local network links, etc, IN ADDITION to URLS.
Kaju-Bubanja commentedon Nov 3, 2022
Sick burn 🗡️ But I also wanted to contribute something useful. Namely https://datatracker.ietf.org/doc/html/draft-thomson-postel-was-wrong-00 @brianjmurrell might be coming from a https://datatracker.ietf.org/doc/html/rfc1122 philosophy but we over time found out that being loose might not be healthy for the ecosystem: https://datatracker.ietf.org/doc/html/draft-thomson-postel-was-wrong-00. Just thought these two rfcs are relevant to both of you.
miigotu commentedon Nov 4, 2022
Exactly. If it was meant to be loose, the validator would be
validator.host, which has a much more loose definition (and much more difficult to validate), rather thanvalidators.url, which has a very strict definition and therefore a trivial path to codifying a validation more effectively. I don't considerurlandhostinterchangeable, and that difference is not mere semantics.I am not normally so brash, however I have a history of bumping heads with this specific OP. I appreciate people who are passionate about proving people wrong and pointing out mistakes, when they are correct. When they commonly are not, I am the opposite of appreciative. 😂
maint: improves `url` module