Here is an example. Given the following list of strings:
1.2.3.4 127.0.0.1 127.0.0.2 127.0.0.11 128.0.0.1 125.0.0.* 200.0.0.1 2001:0db8:85a3:0000:0000:8a2e:0370:1300 2001:0db8:85a3:0000:0000:8a2e:0370:1330 2001:0db8:85a3:0000:0000:8a2e:0370:1337 あいうえお あいうえこ 😃😁😆😐 😃😁😆 😃😁🤣list2regexp will return the following pattern:
^(?:1(?:(?:\.(?:2(?:\.(?:3(?:\.(?:4))))))|(?:2(?:(?:5(?:\.(?:0(?:\.(?:0(?:\.(?:.*)))))))|(?:7(?:\.(?:0(?:\.(?:0(?:\.(?:(?:1(?:|(?:1)))|(?:2))))))))|(?:8(?:\.(?:0(?:\.(?:0(?:\.(?:1)))))))))))|(?:2(?:0(?:0(?:(?:\.(?:0(?:\.(?:0(?:\.(?:1))))))|(?:1(?::(?:0(?:d(?:b(?:8(?::(?:8(?:5(?:a(?:3(?::(?:0(?:0(?:0(?:0(?::(?:0(?:0(?:0(?:0(?::(?:8(?:a(?:2(?:e(?::(?:0(?:3(?:7(?:0(?::(?:1(?:3(?:(?:0(?:0))|(?:3(?:(?:0|7))))))))))))))))))))))))))))))))))))))))))|(?:あ(?:い(?:う(?:え(?:(?:お|こ))))))|(?:😃(?:😁(?:(?:😆(?:|(?:😐)))|(?:🤣))))$it should(could) have newlines for readability. and is genererally an underappreciated feature that leads to people thinking that regex is crazy and unreadable.
I didn’t know about this so I tried testing it out. Yes, this is possible, but requires enabling a specific flag (x when it comes to regexp2) assuming the engine even supports this feature.
also comments. which can be very simple…like metadata about boundary,membership,grouping, or hierarchy.
its supported in most languages except javascript and plain grep.
but it doesnt serve a programmatic use… so even if its a hella useful single char or digit, that massively boosts readability… some people would argue that it’s extra/junk
.*… should do the trick.
Why? 🤔
In my case, to efficiently match against a large number of IP addresses and IP address ranges:
Holy overengineering, Batman. IP addresses of all things are trivially parseable into fixed-length byte sequences. Just do that and use a search tree. Or, IDK; this is an extremely solved problem, there are probably even better solutions out there – but I promise you text regex is involved in none of the good ones.
Even for actual string matching, any decent regex engine will compile and optimise an alternation
a|b|c|...better than this text wrangling will. Unless you have actual benchmarks showing this gives you an advantage with the engine you’re using, you’re just introducing complexity for no gain.Also of note. Regexlicensing.org has issued no licences to date for use of regex in production environments
Not much substance to their argument, mostly just whining. Funny website, though.
It’s called “allowlist” and “denylist” nowadays as it’s easier to understand for non-native speakers and not racist.
I feel like there’s an easier and faster way to match IP addresses than regex, e.g. the search tree apparia suggested. At least use bytes to match 😄
The term’s connection with racism, as well as the value in avoiding its use has been disputed.
https://en.wikipedia.org/wiki/Blacklist_(computing)#Controversy_over_terminology
(See the section that follows for more context.)
Yes, but does it really hurt you to make your software more accessible?
What good arguments do you have for the term blacklist?
“Denylist” is easier to understand and doesn’t spark discussions. Nobody will be offended, whether it’s technically correct to be offended or not.
I’m not a person of colour but your answer still insults me, because it’s so tone deaf.
I think the list of strings with pipes between each element would be shorter than this huge unreadable blob
Being pretty or short is not a goal of this project, nor do I see the value in such goals.
Pretty and short probably also have better performance.
Did you manually come up with a prettier regex to compare against?
Is the goal to match ONLY the given haystack? What would be the use case? Because usually for dynamic usage you need the opposite site, find matches not the pattern. This is basically a pattern finder. Don’t get me wrong, from technical level this is impressive and interesting. But I don’t see a use case right now.
Well I could imagine a scenario to create test cases for your software that uses regex. But the resulting pattern is unreadable and probably not very efficient too. I don’t have a better solution to this, so don’t take my reply as a poison; I just analyze with interest.
I have lots of places I can use regex to match strings, but otherwise have no easy way to say “match any in list”.
Is it stupid? Sure. But I can’t fix the umpteen web consoles I manage; I have to work with them as they are.
(?: strings|stringb|stringc)
Is valid regex given a list of strings to match, no?
Is the advantage here some performance gain or potential to use fewer characters depending on the list?
Edit: I see elsewhere it’s specifically so you can match against a long list of specific IPs - so maybe a bit of both.
I see. Even if I have no use case for it personally, I want you to thank you for sharing. Its probably a niche use case, but always interesting to see what problems others try to solve and how they did it. The code is actually smaller than I expected.
Does it do exclusions? Can it merge regexp?
Those would be interesting features, but it currently only operates on plain text lists, or lists with wildcards (*) which are converted into regexp (.*), or lists with CIDR suffixes (127.0.0.0/24). You can disable escaping, but list2regexp doesn’t understand the individual segments of regular expressions, so the result will probably be wrong.
With that said, the library being used to handle escaping and testing the generated patterns (regexp2) does expose the syntax tree so the information is available to do interesting things with, such as what you suggest.
Merging probably has more value as people would understand the edge cases, with generated regexp it would be harder for someone to confidently understand the edges without having a deeper understanding of the algorithm and data set.
