I am using Nokogiri (with Rails) to try and parse this site and Nokogiri is not cooperating. I made it all the way down the HTML chain using css selectors and everything was working, and then I encountered a plain DIV with no class name or ID. I need to access the content inside this div but cannot do it. Also, in place of this div with no class or ID, Nokogiri is returning me a div with an ID name that is not present anywhere if you inspect the live site.
structure of the HTML document
In my picture, I have made it inside the div with the id="depth-chart-page-wrapper", but I need to get to the div with the class="depth-chart-wall", but it is wrapped in a plain div with no class or id or any attributes, and I cannot find a way to get a hold of this element.
So far this I've used this statement:
This is my Nokogiri scraper in Ruby on Rails
and my result returns me this:
What nokogiri is returning to me
My result is saying that the div with class "block__content" has three children:
- h1 with class "block__title"
- Nokogiri::XML::Text:0x3fc184f8aa18 "\n"
- div with id="rw-depth-chart-value"
But if you look at the structure of the HTML on the site you should only see two:
- h1 with class "block__title"
- the div with no class name or id
As I said above, if you inspect the site you will see there is no element in the whole site with an ID of "rw-depth-chart-value". Why is Nokogiri returning me this value and why can't I access the plain div the same way I've been able to access everything else up to that point?