我想要检索的网页有一个Ajax负载下翻功能阿拉Twitter的元素。出于某种原因,这是不正常工作。我加了一些打印语句来调试它,我总是得到相同的项目数量,然后函数返回。我在做什么错在这里?
WD = webdriver.Firefox()
wd.implicitly_wait(3)
高清get_items(项):
打印LEN(项)
wd.execute_script(window.scrollTo(0,document.body.scrollHeight);)
#LEN(项目)和len(wd.find_elements逐...())这两个似乎总是返回相同的数字
#如果我要开始,而真正的循环:这是可行的,但当然...永远不会结束
而LEN(wd.find_elements_by_class_name(流项目))> LEN(项):
项目= wd.find_elements_by_class_name(流项目)
打印项目
wd.execute_script(window.scrollTo(0,document.body.scrollHeight);)
退换货品
高清测试():
get_page('http://twitter.com/)
get_items(wd.find_elements_by_class_name(流项目))
解决方案
尝试把睡眠之间
WD = webdriver.Firefox()
wd.implicitly_wait(3)
高清get_items(项):
打印LEN(项)
wd.execute_script(window.scrollTo(0,document.body.scrollHeight);)
#LEN(项目)和len(wd.find_elements逐...())这两个似乎总是返回相同的数字
#如果我要开始,而真正的循环:这是可行的,但当然...永远不会结束
睡眠(5)#seconds
而LEN(wd.find_elements_by_class_name(流项目))> LEN(项):
项目= wd.find_elements_by_class_name(流项目)
打印项目
wd.execute_script(window.scrollTo(0,document.body.scrollHeight);)
退换货品
高清测试():
get_page('http://twitter.com/)
get_items(wd.find_elements_by_class_name(流项目))
注:硬盘休眠只是为了证明它的作品。请使用等待包等待一个聪明的条件吧。
I'm trying to retrieve elements in a page that has an ajax-load scroll-down functionality alla Twitter. For some reason this isn't working properly. I added some print statements to debug it and I always get the same amount of items and then the function returns. What am I doing wrong here?
wd = webdriver.Firefox()
wd.implicitly_wait(3)
def get_items(items):
print len(items)
wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# len(items) and len(wd.find_elements-by...()) both always seem to return the same number
# if I were to start the loop with while True: it would work, but of course... never end
while len(wd.find_elements_by_class_name('stream-item')) > len(items):
items = wd.find_elements_by_class_name('stream-item')
print items
wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
return items
def test():
get_page('http://twitter.com/')
get_items(wd.find_elements_by_class_name('stream-item'))
解决方案
Try putting a sleep in between
wd = webdriver.Firefox()
wd.implicitly_wait(3)
def get_items(items):
print len(items)
wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# len(items) and len(wd.find_elements-by...()) both always seem to return the same number
# if I were to start the loop with while True: it would work, but of course... never end
sleep(5) #seconds
while len(wd.find_elements_by_class_name('stream-item')) > len(items):
items = wd.find_elements_by_class_name('stream-item')
print items
wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
return items
def test():
get_page('http://twitter.com/')
get_items(wd.find_elements_by_class_name('stream-item'))
Note: The hard sleep is just for demonstrating that it works. Please use the waits package to wait for a smart condition instead.