As far I know, structural engineering is not a physically tasking profession. Engineering careers offer challenging opportunities so I would suggest that you do further research before choosing your area of concentration. I have also interviewed one of my colleagues who is a structural engineer. Please see below a summary of his response.
In my experience (19 years as a structural engineer with consulting companies), most of my time is "office work." This is broken up by occasional (every week or two) "site visits." These can be just to look at a site prior to design, to observe construction in progress, to look at problems or mistakes that occurred during construction, or to do a final review of the completed project.
In any case, physical requirements are minimal. A pencil, a set of plans, a camera, and a tape measure are all I usually take. Ladders and so on that might be required at the site are generally provided by the Contractor, and usually set up by their crew (often this is required by their insurance company).
Hope this helps.