Re: [FE-discuss] Passing unicode to Email validator

Top Page
Author: Leandro Lucarella
Date:  
To: formencode-discuss
Subject: Re: [FE-discuss] Passing unicode to Email validator
Leandro Lucarella, el 15 de octubre a las 15:39 me escribiste:
> Ian Bicking, el 15 de octubre a las 10:49 me escribiste:
> > > Should Email validator first check if the domain is valid ASCII? (at least
> > > if IDN[1] are not supported by pyDNS).
> >
> > Yes, that seems reasonable. I suppose it could also use the punycode
> > encoding. I think that all IDNs are is punycode-encoded unicode, right?
>
> This is what Wikipedia says at least =)
>
> Here is a patch that implements IDNA support.
>
> > > Should I send a patch? Or you think this is a pyDNS bug?
> > >
> > > [1] http://en.wikipedia.org/wiki/Internationalized_domain_name
> >
> > Yes, probably pyDNS should handle this more gracefully as well.
>
> I'll try to contact pyDNS author to see if he can add IDNA support to it.


Here is an alternative patch that uses dnspython[1] (which supports IDN)
instead of pyDNS.

[1] http://www.dnspython.org/

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Si ella es la flor, yo soy la espina
Si ella es quien florece, yo quien se marchita
Y estamos en eclipse total, y estamos en eclipse total
Completamente cruzados, completamente cruzados
>From c557820b95a13d847f9b20f2e084c0cc70a604a7 Mon Sep 17 00:00:00 2001
From: Leandro Lucarella <llucarella@???>
Date: Wed, 15 Oct 2008 16:09:44 -0300
Subject: [PATCH] Use dnspython to support IDN when using domain resolution in Email validator

When trying to resolve a domain (via the resolve_domain option for the
Email validator), and the e-mail address is a unicode string (even when it
only contains ASCII characters) pyDNS goes crazy, and raises almost random
unicode errors. So use dnspython[1] instead, that support IDN since 2005.

This patch does some identation fixes too.

[1] http://www.dnspython.org
---
 formencode/validators.py |   55 ++++++++++++++++++++++++++-------------------
 1 files changed, 32 insertions(+), 23 deletions(-)

diff --git a/formencode/validators.py b/formencode/validators.py
index 04587b9..4afce34 100644
--- a/formencode/validators.py
+++ b/formencode/validators.py
@@ -46,8 +46,9 @@ import cgi
 import fieldstorage

 try:
- import DNS
- DNS.DiscoverNameServers()
+ import dns.resolver
+ import dns.exception
+ from encodings import idna
     have_dns=True
 except ImportError:
     have_dns=False
@@ -1269,8 +1270,8 @@ class Email(FancyValidator):

     If you pass ``resolve_domain=True``, then it will try to resolve
     the domain name to make sure it's valid.  This takes longer, of
- course. You must have the `pyDNS <http://pydns.sf.net>`__ modules
- installed to look up DNS (MX and A) records.
+ course. You must have the `dnspython <http://www.dnspython.org/>`__
+ modules installed to look up DNS (MX and A) records.

     ::

@@ -1309,10 +1310,16 @@ class Email(FancyValidator):
         >>> e.to_python('test@???')
         'test@???'
         >>> # NOTE: If you do not have PyDNS installed this example won't work:
+ >>> e.to_python((u'test@t\u016bdali\u0146.lv')
+ u'test@t\u016bdali\u0146.lv'
         >>> e.to_python('test@???')
         Traceback (most recent call last):
             ...
- Invalid: The domain of the email address does not exist (the portion after the @: thisdomaindoesnotexistithinkforsure.com)
+ Invalid: The domain of the email address does not exist (the portion afte
+ >>> e.to_python(u'test@thisdomaindoesnotexistithinkforsure\xe1.com')
+ Traceback (most recent call last):
+ ...
+ Invalid: The domain of the email address does not exist (the portion afte
         >>> e = Email(not_empty=False)
         >>> e.to_python('')

@@ -1341,10 +1348,10 @@ class Email(FancyValidator):
             if not have_dns:
                 import warnings
                 warnings.warn(
- "pyDNS <http://pydns.sf.net> is not installed on "
- "your system (or the DNS package cannot be found). "
- "I cannot resolve domain names in addresses")
- raise ImportError, "no module named DNS"
+ "dnspython <http://www.dnspython.org/> is not installed "
+ "on your system (or the dns.resolver package cannot be "
+ "found). I cannot resolve domain names in addresses")
+ raise ImportError, "no module named dns.resolver"

     def validate_python(self, value, state):
         if not value:
@@ -1364,26 +1371,28 @@ class Email(FancyValidator):
                 self.message('badUsername', state,
                              username=username),
                 value, state)
- if not self.domainRE.search(domain):
+ idna_domain = '.'.join([idna.ToASCII(l) for l in domain.split('.')])
+ if not self.domainRE.search(idna_domain):
             raise Invalid(
                 self.message('badDomain', state,
                              domain=domain),
                 value, state)
         if self.resolve_domain:
- assert have_dns, "pyDNS should be available"
- try:
- a=DNS.DnsRequest(domain, qtype='mx').req().answers
- if not a:
- a=DNS.DnsRequest(domain, qtype='a').req().answers
- dnsdomains=[x['data'] for x in a]
- except (socket.error, DNS.DNSError), e:
+ assert have_dns, "dnspython should be available"
+ try:
+ try:
+ a = dns.resolver.query(domain, 'MX')
+ except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer), e:
+ try:
+ a = dns.resolver.query(domain, 'A')
+ except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer), e:
+ raise Invalid(
+ self.message('domainDoesNotExist', state,
+ domain=domain),
+ value, state)
+ except (socket.error, dns.exception.DNSException), e:
                 raise Invalid(
- self.message('socketError', state, error=e),
- value, state)
- if not dnsdomains:
- raise Invalid(
- self.message('domainDoesNotExist', state,
- domain=domain),
+ self.message('socketError', state, error=e),
                     value, state)

     def _to_python(self, value, state):
-- 
1.5.6.5

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________
FormEncode-discuss mailing list
FormEncode-discuss@???
https://lists.sourceforge.net/lists/listinfo/formencode-discuss