puppeteer的使用
1、puppeteer配置安装
2、puppeteer元素基本操作-输入文本与元素点击
3、puppeteer获取文本元素值
4、puppeteer处理多个元素
5、pupputeer切换iframe进行安居客登陆操作
6、puppeteer拖拽操作阿里云验证码
7、puppeteer自动抓取网页内容,然后自动发一条微博
8、puppeteer模拟快捷键
9、puppeteer切换浏览器tab页
10、puppeteer处理弹出对话框
11、puppeteer执行JavaScript方法
12、配置typescript环境(puppeteer包已经安装)
1、puppeteer配置安装 <--返回
首先需要node环境;然后安装puppeteer(同时会安装Chromium)
npm i puppeteer
参考:https://www.cnblogs.com/xy-ouyang/p/12167226.html
2、puppeteer元素基本操作-输入文本与元素点击 <--返回
// puppeteer元素基本操作-输入文本与元素点击 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 } }) const page = await browser.newPage() await page.goto('https://www.baidu.com') const input_area = await page.$('#kw') // 定位输入框 await input_area.type('hello world') // 输入文本 // const search_btn = page.$('#su') // 定位'百度一下'搜索按钮 // await search_btn.click() // 点击 await page.click('#su') } fun()
3、puppeteer获取文本元素值 <--返回
// puppeteer获取文本元素值 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 } }) const page = await browser.newPage() await page.goto('https://www.baidu.com') const input_area = await page.$('#kw') // 定位输入框 await input_area.type('hello world') // 输入文本 // const search_btn = page.$('#su') // 定位'百度一下'搜索按钮 // await search_btn.click() // 点击 await page.click('#su') await page.waitForSelector('div#content_left > div.result-op.c-container.xpath-log') let resultText = await page.$eval('div#content_left > div.result-op.c-container.xpath-log', ele => ele.innerHTML) console.log(`resultText=${resultText}`) } fun()
4、puppeteer处理多个元素 <--返回
// puppeteer处理多个元素 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 } }) const page = await browser.newPage() await page.goto('https://www.jd.com') const input = await page.$('#key') // 定位输入框 await input.type('手机') // 输入文本 await page.keyboard.press('Enter') // 按下回车键 await page.waitForSelector('ul.gl-warp > li') let resultTextList = await page.$$eval('ul.gl-warp > li', eles => eles.map(ele => ele.innerText)) console.log('resultTextList = ', resultTextList) } fun()
5、pupputeer切换iframe进行安居客登陆操作 <--返回
// pupputeer切换iframe进行安居客登陆操作 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 } }) const page = await browser.newPage() await page.goto('https://login.anjuke.com/login/form') // 打印页面所有的frame的地址 await page.frames().map(frame => { console.log(frame.url()) }) // 通过frame的url定位到frame const targetFrameUrl = 'https://login.anjuke.com/login/iframeform' const frame = await page.frames().find(frame => frame.url().includes(targetFrameUrl)) const phone = await frame.waitForSelector('#phoneIpt') await phone.type('13530125464') } fun()
6、puppeteer拖拽操作阿里云验证码 <--返回
// puppeteer拖拽操作阿里云验证码 async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 }, ingoreDefaultArgs: ['--enable-automation'] }) const page = await browser.newPage() await page.goto('https://account.aliyun.com/register/register.htm', { waitUntil: 'networkidle2' }) // 等待页面加载完 // 定位到frame const frame = await page.frames().find(frame => frame.url().includes('https://passport.aliyun.com')) // 定位到验证滑块 const span = await frame.waitForSelector('#nc_1_nlz') const spanInfo = await span.boundingBox() console.log(spanInfo) const div = await frame.waitForSelector('div#nc_1_scale_text > span') const divInfo = await div.boundingBox() await page.mouse.move(spanInfo.x, spanInfo.y) await page.mouse.down() // 鼠标移动 for (let i = 0, width = divInfo.width; i < width; i++) { await page.mouse.move(spanInfo.x + i, spanInfo.y) } // 松开鼠标 await page.mouse.up() } fun()
7、puppeteer自动抓取网页内容,然后自动发一条微博 <--返回
// puppeteer自动抓取网页内容,然后自动发一条微博 const puppeteer = require('puppeteer') const username = 'xxxxxxxx' const password = 'xxxxxxxx' async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 }, ignoreDefaultArgs: ['enable-automation'], slowMo: 200, args: ['--window-size=1366,768'] }) const page = await browser.newPage() // 抓取网页内容文本 await page.goto('http://wufazhuce.com', { waitUntil: 'networkidle2'}) const oneCita = await page.waitForSelector('div.fp-one-cita-wrapper > div.fp-one-cita > a') const oneText = await page.$eval('div.fp-one-cita-wrapper > div.fp-one-cita > a', ele => ele.innerHTML) console.log('oneText: ', oneText) /* 自动发一条微博 */ await page.goto('https://weibo.com/', { waitUntil: 'networkidle2'}) await page.waitFor(2000) await page.reload() await page.waitFor(5000) // 输入用户名 const usernameInput = await page.waitForSelector('div.input_wrap > #loginname') await usernameInput.click() await usernameInput.type(username) // 输入密码 const passwordInput = await page.waitForSelector('input[type="password"]') await passwordInput.click() await passwordInput.type(password) // 点击'登陆'按钮 const loginBtn = await page.waitForSelector('a[action-type="btn_submit"]') await loginBtn.click() // 输入发表微博内容 const textArea = await page.waitForSelector('textarea[class="W_input"]') await textArea.click() await textArea.type(oneText) // 点击'发布'按钮 const sendBtn = await page.waitForSelector('a[node-type="submit"]') await sendBtn.click() } fun()
8、puppeteer模拟快捷键 <--返回
// puppeteer模拟快捷键 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 }, ignoreDefaultArgs: ['enable-automation'], slowMo: 200, args: ['--window-size=1366,768'] }) const page = await browser.newPage() await page.goto('http://39.107.96.138:3000/signin', { waitUntil: 'networkidle2' }) const username = await page.waitForSelector('#name') await username.type('user1') const pwd = await page.waitForSelector('#pass') await pwd.type('123456') const loginBtn = await page.waitForSelector('.span-primary') await loginBtn.click() const createTopicBtn = await page.waitForSelector('#create_topic_btn') await createTopicBtn.click() const textarea = await page.waitForSelector('.CodeMirror-code') await textarea.click() await page.keyboard.type('hello world') // ctrl + a await page.keyboard.down('Control')
await page.keyboard.press('a') //await page.keyboard.down('a') //await page.keyboard.up('a') await page.keyboard.up('Control') // ctrl + b await page.keyboard.down('Control')
await page.keyboard.press('b') //await page.keyboard.down('b') //await page.keyboard.up('b') await page.keyboard.up('Control') } fun()
9、puppeteer切换浏览器tab页 <--返回
// puppeteer切换浏览器tab页 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 }, // ignoreDefaultArgs: ['enable-automation'], // slowMo: 200, args: ['--window-size=1366,768'] }) const page = await browser.newPage() await page.goto('http://music.taihe.com/', { waitUntil: 'networkidle2' }) // 关闭广告 const ad = await page.waitForSelector('#__qianqian_pop_closebtn') await ad.click() // 点击'链接',打开新窗口 const link = await page.waitForSelector('div.mod-tag-box>h3') await link.click() // 获取到新窗口 const target = await browser.waitForTarget(target => target.url().includes('tag')) const newPage = await target.page() // 等待新窗口加载完成 await newPage.waitForSelector('ul[select="20"]') const text = await newPage.$eval('ul[select="20"]', ele => ele.innerText) console.log(text) } fun()
10、puppeteer处理弹出对话框 <--返回
index.html
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta http-equiv="X-UA-Compatible" content="ie=edge"> <title>Document</title> </head> <body> <p>点击显示一个确认对话框.</p> <button id='btn1' onclick="myFunction1()">点击</button> <p>点击显示一个输入文本的对话框.</p> <button id="btn2" onclick="myFunction2()">Try it</button> <p id="demo"></p> <script> function myFunction1() { confirm("Press a button!"); } function myFunction2() { var person = prompt("Please enter your name", "Harry Potter"); if (person != null) { document.getElementById("demo").innerHTML = "Hello " + person + "! How are you today?"; } } </script> </body> </html>
---
// puppeteer处理弹出对话框 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 }, ignoreDefaultArgs: ['enable-automation'], slowMo: 200, args: ['--window-size=1366,768'] }) const page = await browser.newPage() await page.goto('file:///C:/Users/oy/Desktop/index.html', { waitUntil: 'networkidle2' }) page.on('dialog', async dialog => { console.log(dialog.type()) console.log(dialog.message()) await dialog.accept('haha') // 点击'确认' }) const btn1 = await page.waitForSelector('#btn1') await btn1.click() const btn2 = await page.waitForSelector('#btn2') await btn2.click() } fun()
11、puppeteer执行JavaScript方法 <--返回
// puppeteer执行JavaScript方法 const puppeteer = require('puppeteer') async function fun() { const browser = await puppeteer.launch({ headless: false, defaultViewport: { width: 1366, height: 768 }, ignoreDefaultArgs: ['enable-automation'], slowMo: 200, args: ['--window-size=1366,768'] }) const page = await browser.newPage() await page.goto('https://www.ctrip.com/', { waitUntil: 'networkidle2' }) await page.evaluate(() => { document.querySelector('#HD_CheckIn').value = '2020-xx-xx' }) } fun()
12、配置typescript环境(puppeteer包已经安装) <--返回
1) 安装typescript
npm i typescript -g
2) 执行tsc -version命令
3)npm i @types/node -D
4) npm i @types/puppeteer -D
5)项目根目录下执行 tsc --init 命令, 会在项目根目录下生成 tsconfig.json 文件
6)测试代码
TestAction.ts
import { Page, Browser, launch } from "puppeteer"; class TestAction { public gotoUrl(page: Page, url: string) { page.goto(url, { waitUntil: 'networkidle2' }) } } async function main() { const browser: Browser = await launch({ headless: false }) const page: Page = await browser.newPage() const testAction = new TestAction() testAction.gotoUrl(page, 'https://www.baidu.com/') } main()
7)执行tsc命令,会在项目根目录下生成 out/actions/TestAction.js 文件
8)执行命令
node .\out\actoins\TestAction.js
---
posted on 2020-01-31 00:39 wenbin_ouyang 阅读(2157) 评论(0) 编辑 收藏 举报