Using Selenium to parse a page for links that do not exist in the source

Tue Aug 26, 2014 6:43 am

I am looking to find a way to code a script in python that can extract the link to each item in my Steam inventory. Unfortunately I cannot link it but use the image below as reference. Essentially each item in the inventory can be clicked and when clicked a different description will be shown on the right hand side.

What it should do is be able to extract the href link from the blue highlighted field in the image below:

Using Selenium to parse a page for links that do not exist in the source Ln3YUXQ

Using Selenium to parse a page for links that do not exist in the source Ln3YUXQ

So it should be able to access each "item holder" class and find the value attached to the attribute href. After that it should be able to extract the text on the right side of the screen. So in the image example above, it should be able to see the "View applicable games" button, see the text "Tags: Tradable, Not Marketable", etc.

I have done some research and I know how to grab the source from the page or get links based on what attribute it has etc... However, the issue is that the links I want to extract are not even in the source. I'm guessing that it doesn't work because the source changes based on which item you click in the inventory...? I talked to some people and they have told me that it isn't even possible to do this in Selenium... Is this true? If so does anyone have recommendations on something else I can use.