python - How to extract all the hrefs and titles from several `<a href="" title=""> tags? -


given file:

<a data-parent="#accordion1" data-toggle="collapse" href="# fruitname1" title="click expand drug name"> <span class="list-unstyled" style="text-decoration: none;"></span> glipizide           </a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114223" title="click view lemons (lemons) | poq  #114223 | box;67 pz | presentation | fruit company 1 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114226" title="click view lemons (lemons) | poq  #114226 | box;67 pz | presentation | fruit company 2 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114305" title="click view lemons (lemons) | poq  #114305 | box;67 pz | presentation | fruit company 3 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114370" title="click view lemons (lemons) | poq  #114370 | box;67 pz | discontinued | fruit company 1 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114378" title="click view lemons (lemons) | poq  #114378 | box;67 pz | discontinued | fruit company 4 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114387" title="click view lemons (lemons) | poq  #114387 | box;67 pz | discontinued | fruit company 5 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114438" title="click view lemons (lemons) | poq  #114438 | box;67 pz | presentation | fruit company 2 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114497" title="click view lemons (lemons) | poq  #114497 | box;67 pz | presentation | fruit company 5 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114542" title="click view lemons (lemons) | poq  #114542 | box;67 pz | discontinued | fruit company 3 ">                               lemons (lemons)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114550" title="click view lemons (lemons) | poq  #114550 |           </a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=117270" title="click view grapes (green grapes ; aus) | poq  #117270 | box;67 pz | presentation | fruit company 10  ">                               grapes (green grapes ; aus)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=117511" title="click view grapes (green grapes ; aus) | poq  #117511 | box;67 pz | presentation | fruit company 11 ">                               grapes (green grapes ; aus)</a> <a href="/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=117620" title="click view grapes (green grapes ; aus) | poq  #117620 | box;67 pz | presentation | fruit company 12 ">

using regex or beautiful soup, how extract <a href="" title="">, adding www.example.com before href tags into:

www.example.com/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114223 |  title= | click view lemons (lemons) | poq  #114223 | box;67 pz | presentation | fruit company 1 | lemons (lemons) www.example.com/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114226 |  title= | click view lemons (lemons) | poq  #114226 | box;67 pz | presentation | fruit company 2 | lemons (lemons) www.example.com/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114305 |  title= | click view lemons (lemons) | poq  #114305 | box;67 pz | presentation | fruit company 3 | lemons (lemons) www.example.com/loads/data/usersindex.cfm?event=overview.subprocess&amp;applno=114370 |  title= | click view lemons (lemons) | poq  #114370 | box;67 pz | discontinued | fruit company 1 | lemons (lemons)

i tried to:

for in soup.tbody.findall('a', href=true):     r = re.compile('(?<=href=").*?(?=")')     r.findall(str(a)

and:

for in soup.tbody.findall('a', href=true):     print (a.find('a')['href'])     print (a.find('a')['title'])

however, not how rearrange titles , hrefs. update

based on odradek's answer, tried this:

soup = beautifulsoup(open('file.htm'), 'lxml') in soup.tbody.findall('a', href=true):     html =     prefix = 'www.example.com'     template = '{prefix}{url} | {title}'.format     links = [template(prefix=prefix, url=e['href'], title=e['title']) e in html.find_all('a', href=true)]     print(links)

however got:

[] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] []

you can use beautifulsoup parsing methods instead of complicated regexp this:

# url want add @ beginning prefix = 'www.example.com'  # template of desired output template = '{prefix}{url} | {title}'.format  # resulting list, please note "html" variable # given source code. links = [template(prefix=prefix, url=e.get('href'), title=e.get('title'))          e in html.find_all('a', href=true)]

when ran against 2 a tags of list:

$ python get_all_a.py www.example.com/loads/data/usersindex.cfm?event=overview.subprocess&applno=117511 | click view grapes (green grapes ; aus) | poq  #117511 | box;67 pz | presentation | fruit company 11  www.example.com/loads/data/usersindex.cfm?event=overview.subprocess&applno=117620 | click view grapes (green grapes ; aus) | poq  #117620 | box;67 pz | presentation | fruit company 12

based on update, shouldn't put piece of code inside loop, rather:

html = beautifulsoup(open('file.htm'), 'html.parser')  prefix = 'www.example.com'  template = '{prefix}{url} | {title}'.format  # inside list comprehension loop implied links = [template(prefix=prefix, url=e.get('href'), title=e.get('title'))          e in html.find_all('a', href=true)]

Comments

Post a Comment

Popular posts from this blog

How to understand 2 main() functions after using uftrace to profile the C++ program? -

i trying uftrace profile following simple c++ program: #include <iostream> class { public: a() {std::cout << "a created" << std::endl;} ~a() {std::cout << "a destroyed" << std::endl;} }; int main() { a; return 0; } the profile result this: # uftrace a.out created destroyed # duration tid function 2.026 [ 4828] | __cxa_atexit(); [ 4828] | main() { [ 4828] | __static_initialization_and_destruction_0() { 89.397 [ 4828] | std::ios_base::init::init(); 0.768 [ 4828] | __cxa_atexit(); 93.029 [ 4828] | } /* __static_initialization_and_destruction_0 */ 94.425 [ 4828] | } /* main */ [ 4828] | main() { [ 4828] | a::a() { 11.104 [ 4828] | std::operator <<(); 10.825 [ 4828] | std::basic_ostream::operator <<(); 24.514 [ 4828] | } /* a::a */ [ 4828] | a::~a() { 0.978 [ 4828] | ...
Read more

c# - Update a combobox from a presenter (MVP) -

i using mvp in project, new in mvp. have 2 comboboxes. when select option in combobox, combobox should filled new data. this action in presenter. view 'view1' in presenter, , introduced combobox1 , combobox2 properties in 'view1', because need 'datasource', 'displaymember', 'valuemember' , 'refresh()' in method below. but, when using pattern, enough send property like public string combobox2 { { return combobox1.selectedvalue.tosstring(); } } into presenter not whole combobox. how can solve problem? public void onselectedindexchangedcombobox1() { if (view1.combobox1.selectedindex == -1) { return; } datatable dt = tools.getdatatable("a path"); var query = (from o in dt.asenumerable() o.field<string>("afield") == farmerview.combobox1.selectedvalue.tostring() orderby o.field<string>("anotherfield") select...
Read more

How to put a lock and transaction on table using spring 4 or above using jdbcTemplate and annotations like @Transactional? -

i trying put lock on table while writing on table , if happened in between roll-back . trying convert below code lock table test_g1 read; lock table test_g write; -- begin; start transaction; insert test_g1 values(143); insert test_g values(145); select * test_g1; select * test_g; rollback; select * test_g; unlock tables; how convert above code @transactional spring jdbctemplate code? @transactional(rollbackfor={dataaccessexception.class}) public void test(){ jdbctemplate.execute("insert test1 (id, nam) values (4, 'a')"); throw new dataaccessexception("error") { }; } here trying throw error, insert statement should rollback it's not happening . thanks edit-1 attaching code , doing in jdbcdaoimpl.java , mentioned problem comment above test() . app.java package com.cgiri.javabrains.spring4; import org.springframework.beans.factory.annotation.autowired; import org.springframework.context.applic...
Read more