python爬虫中遇到“\xb5”、“xa0”等字符时报错编码错误的处理方式
写python爬虫是遇到编码错误
报错为：
UnicodeEncodeError: 'gbk' codec can't encode character '\xa0'
经过多方查找发现 \xa0是html网页源码中的空格
解决方法
替换掉字符 ：replace(u'\xa0', u' ')
下面是一些html中的常见符号
|
chr |
HexCode |
Numeric |
HTML entity |
|
" |
\x22 |
" |
" |
|
& |
\x26 |
& |
& |
|
< |
\x3C |
< |
< |
|
> |
\x3E |
> |
> |
|
空格 |
\xA0 |
  |
|
|
¡ |
\xA1 |
¡ |
¡ |
|
¢ |
\xA2 |
¢ |
¢ |
|
£ |
\xA3 |
£ |
£ |
|
¤ |
\xA4 |
¤ |
¤ |
|
¥ |
\xA5 |
¥ |
¥ |
|
¦ |
\xA6 |
¦ |
¦ |
|
§ |
\xA7 |
§ |
§ |
|
¨ |
\xA8 |
¨ |
¨ |
|
© |
\xA9 |
© |
© |
|
ª |
\xAA |
ª |
ª |
|
« |
\xAB |
« |
« |
|
¬ |
\xAC |
¬ |
¬ |
|
|
\xAD |
­ |
­ |
|
® |
\xAE |
® |
® |
|
¯ |
\xAF |
¯ |
¯ |
|
° |
\xB0 |
° |
° |
|
± |
\xB1 |
± |
± |
|
² |
\xB2 |
² |
² |
|
³ |
\xB3 |
³ |
³ |
|
´ |
\xB4 |
´ |
´ |
|
µ |
\xB5 |
µ |
µ |
|
¶ |
\xB6 |
¶ |
¶ |
|
· |
\xB7 |
· |
· |
|
¸ |
\xB8 |
¸ |
¸ |
|
¹ |
\xB9 |
¹ |
¹ |
|
º |
\xBA |
º |
º |
|
» |
\xBB |
» |
» |
|
¼ |
\xBC |
¼ |
¼ |
|
½ |
\xBD |
½ |
½ |
|
¾ |
\xBE |
¾ |
¾ |
|
¿ |
\xBF |
¿ |
¿ |
|
× |
\xD7 |
× |
× |
|
÷ |
\xF7 |
÷ |
÷ |
|
ƒ |
\u0192 |
ƒ |
ƒ |
|
ˆ |
\u02C6 |
ˆ |
ˆ |
|
˜ |
\u02DC |
˜ |
˜ |
|
|
\u2002 |
  |
  |
|
|
\u2003 |
  |
  |
|
|
\u2009 |
  |
  |
|
|
\u200C |
‌ |
‌ |
|
|
\u200D |
‍ |
‍ |
|
|
\u200E |
‎ |
‎ |
|
|
\u200F |
‏ |
‏ |
|
– |
\u2013 |
– |
– |
|
— |
\u2014 |
— |
— |
|
‘ |
\u2018 |
‘ |
‘ |
|
’ |
\u2019 |
’ |
’ |
|
‚ |
\u201A |
‚ |
‚ |
|
“ |
\u201C |
“ |
“ |
|
” |
\u201D |
” |
” |
|
„ |
\u201E |
„ |
„ |
|
† |
\u2020 |
† |
† |
|
‡ |
\u2021 |
‡ |
‡ |
|
• |
\u2022 |
• |
• |
|
… |
\u2026 |
… |
… |
|
‰ |
\u2030 |
‰ |
‰ |
|
′ |
\u2032 |
′ |
′ |
|
″ |
\u2033 |
″ |
″ |
|
‹ |
\u2039 |
‹ |
‹ |
|
› |
\u203A |
› |
› |
|
‾ |
\u203E |
‾ |
‾ |
|
⁄ |
\u2044 |
⁄ |
⁄ |
|
€ |
\u20AC |
€ |
€ |
|
ℑ |
\u2111 |
ℑ |
ℑ |
|
? |
\u2113 |
ℓ |
|
|
№ |
\u2116 |
№ |
|
|
℘ |
\u2118 |
℘ |
℘ |
|
ℜ |
\u211C |
ℜ |
ℜ |
|
™ |
\u2122 |
™ |
™ |
|
ℵ |
\u2135 |
ℵ |
ℵ |
|
← |
\u2190 |
← |
← |
|
↑ |
\u2191 |
↑ |
↑ |
|
→ |
\u2192 |
→ |
→ |
|
↓ |
\u2193 |
↓ |
↓ |
|
↔ |
\u2194 |
↔ |
↔ |
|
↵ |
\u21B5 |
↵ |
↵ |
|
⇐ |
\u21D0 |
⇐ |
⇐ |
|
⇑ |
\u21D1 |
⇑ |
⇑ |
|
⇒ |
\u21D2 |
⇒ |
⇒ |
|
⇓ |
\u21D3 |
⇓ |
⇓ |
|
⇔ |
\u21D4 |
⇔ |
⇔ |
|
∀ |
\u2200 |
∀ |
∀ |
|
∂ |
\u2202 |
∂ |
∂ |
|
∃ |
\u2203 |
∃ |
∃ |
|
∅ |
\u2205 |
∅ |
∅ |
|
∇ |
\u2207 |
∇ |
∇ |
|
∈ |
\u2208 |
∈ |
∈ |
|
∉ |
\u2209 |
∉ |
∉ |
|
∋ |
\u220B |
∋ |
∋ |
|
∏ |
\u220F |
∏ |
∏ |
|
∑ |
\u2211 |
∑ |
∑ |
|
− |
\u2212 |
− |
− |
|
∗ |
\u2217 |
∗ |
∗ |
|
√ |
\u221A |
√ |
√ |
|
∝ |
\u221D |
∝ |
∝ |
|
∞ |
\u221E |
∞ |
∞ |
|
∠ |
\u2220 |
∠ |
∠ |
|
∧ |
\u2227 |
∧ |
∧ |
|
∨ |
\u2228 |
∨ |
∨ |
|
∩ |
\u2229 |
∩ |
∩ |
|
∪ |
\u222A |
∪ |
∪ |
|
∫ |
\u222B |
∫ |
∫ |
|
∴ |
\u2234 |
∴ |
∴ |
|
∼ |
\u223C |
∼ |
∼ |
|
≅ |
\u2245 |
≅ |
≅ |
|
≈ |
\u2248 |
≈ |
≈ |
|
≠ |
\u2260 |
≠ |
≠ |
|
≡ |
\u2261 |
≡ |
≡ |
|
≤ |
\u2264 |
≤ |
≤ |
|
≥ |
\u2265 |
≥ |
≥ |
|
⊂ |
\u2282 |
⊂ |
⊂ |
|
⊃ |
\u2283 |
⊃ |
⊃ |
|
⊄ |
\u2284 |
⊄ |
⊄ |
|
⊆ |
\u2286 |
⊆ |
⊆ |
|
⊇ |
\u2287 |
⊇ |
⊇ |
|
⊕ |
\u2295 |
⊕ |
⊕ |
|
⊗ |
\u2297 |
⊗ |
⊗ |
|
⊥ |
\u22A5 |
⊥ |
⊥ |
|
⋅ |
\u22C5 |
⋅ |
⋅ |
|
⌈ |
\u2308 |
⌈ |
⌈ |
|
⌉ |
\u2309 |
⌉ |
⌉ |
|
⌊ |
\u230A |
⌊ |
⌊ |
|
⌋ |
\u230B |
⌋ |
⌋ |
|
⟨ |
\u2329 |
〈 |
⟨ |
|
⟩ |
\u232A |
〉 |
⟩ |
|
◊ |
\u25CA |
◊ |
◊ |
|
♠ |
\u2660 |
♠ |
♠ |
|
♣ |
\u2663 |
♣ |
♣ |
|
♥ |
\u2665 |
♥ |
♥ |
|
♦ |
\u2666 |
♦ |
♦ |
