Unable to view step log on a "local" workspace
We've upgraded our test environment from ElectricCommander 4.2 to 7.3 and are currently running test before we upgrade our Pro environment.
One user is having trouble viewing logs from jobs running on their agents. Clicking on the step logs we see:
The agent returned error 404. Error contacting [their resource]: org.apache.http.ConnectionClosedException: Connection closed unexpectedly
The logs are there in their workspace and can be viewed manually, but not from the Web UI.
Details about our setup:
For testing, I exported/imported their existing project, resources, and pools from our current Pro instance (4.2) to our test instance (7.3). Everything is working on the 4.2 instance.
I've already confirmed the server names are correct in the server and apache config files. I can also view remote logs on another agent in one of my test projects, so it looks like its specific with this user's resources.
Their jagent.log shows the request to get the log file, but it's showing a "file not found" error when trying to fetch the file.
Something to note about their workspace: They are on a windows vm with a fileserver mounted on a separate drive for their workspace. We had to change firewall rules to connect to their lab, and file-sharing rules are hard to get approved, so this was the route we took. From ElectricCommander/ElectricFlow's perspective, the workspace is local to the agent. And this works fine on our 4.2 version.
When investigating an issue like this one, it's very helpful if we understand how a step log file is found and fetched. [Local Workspaces](http://docs.electric-cloud.com/eflow_doc/9_0/User/HTML/FlowUserGuide_9_0.htm#eflow_user/workspace.htm#Local_workspaces) explains this in details.
The next step is checking the logs from the relevant nodes (the Flow server, the agent that run the step and the agent on the web server).
From the server log, we can see that when a step is created (requested to run on an agent), everything seems correct, please notice "winDrive" in the following request:
2019-05-13T18:52:11.216 | DEBUG | Agent-002 | | | | AgentRequestFactoryImpl | onSubmit: request[id=run-265b6ad0-75b0-11e9-83af-0acc96f90ee4-26e052bd-75b0-11e9-bd66-0acc96f90ee4,agent[id=00000000-0000-0000-0000-000000001925,dfe-vmbuild-3.idp.boi.rd.hpicorp.net:0,secure,state=alive,version=4.2.4.71224,protocol=6,os=Microsoft Windows [Version 10.0.14393],platform=windows,secure,serial=1790,signature=jrW1c14u3vY9chyvkac2SQ==,zone[name=default,id=00000000-0000-0000-0000-000000000001,TRANSIENT]]]
0
y:\DFE_Build_Library-8.0.0_MASTER_Build-TEST-17303983
DFE_workspace
D:/workspace/dfe-component-build-tool_63613
build.bat BUILD-PHASE
exec-build-tool- BUI.26e052bd-75b0-11e9-bd66-0acc96f90ee4.log
Also, in the agent log, we can confirm that the data is correct in the request (a different one but uses the same workspace) that the agent receives to run a step/command:
2019-05-23T10:38:23.897 | DEBUG | AgentHttp-007 | | | n-0e101315-7d79-11e9-8d6f-0645b93d2238-0e1da77a-7d79-11e9-bc46-0645b93d2238 | AgentProtocolHandler | Received agent request:
0
Y:\DFE_Build-dfe-component-build_current-core-8.0-18453944
DFE_workspace
ectool setProperty /myCall/resourceToUse dfe-vmbuild-4 ectool setProperty /myCall/BUILD "true"
init-electric-cloud.0e1da77a-7d79-11e9-bc46-0645b93d2238.log
init-electric-cloud.0e1da77a-7d79-11e9-bc46-0645b93d2238-postp.log
However, when the server is asked to resolve a step log file, the data from the server seems wrong. Please notice "windowsJobDirectory" in the following response, "" is unexpected:
2019-05-23T16:38:33.636 | DEBUG | Api-004 | 70446 | 0e101315-7d79-11e9-8d6f-0645b93d2238 | resolveFile | XmlApiRequestHandler | Response for id=14086:
artifacts
https://dfe-vmbuild-4.idp.boi.rd.hpicorp.net:7800/
init-svn-change-chec.0e1e1cac-7d79-11e9-bc46-0645b93d2238.log
Y:DFE_Build-dfe-component-build_current-core-8.0-18453944
When the agent receives such a request to fetch a file like the following, it fails to find the file as the value of "windowsJobDirectory" is wrong:
2019-05-23T10:38:33.808 | DEBUG | AgentHttp-008 | | | fetchFile-268 | AgentProtocolHandler | Received agent request:
Y:?DFE_Build-dfe-component-build_current-core-8.0-18453944
artifacts
init-svn-change-chec.0e1e1cac-7d79-11e9-bc46-0645b93d2238.log
init-svn-change-chec.0e1e1cac-7d79-11e9-bc46-0645b93d2238.log
2019-05-23T10:38:33.808 | DEBUG | AgentHttp-008 | | | fetchFile-268 | AgentProtocolHandler | final path of file to retrieve: Y:?DFE_Build-dfe-component-build_current-core-8.0-18453944\init-svn-change-chec.0e1e1cac-7d79-11e9-bc46-0645b93d2238.log
2019-05-23T10:38:33.808 | DEBUG | AgentHttp-008 | | | fetchFile-268 | FileTransferHelper | Preparing to transfer Y:\\?DFE_Build-dfe-component-build_current-core-8.0-18453944\init-svn-change-chec.0e1e1cac-7d79-11e9-bc46-0645b93d2238.log
2019-05-23T10:38:33.808 | DEBUG | AgentHttp-008 | | | fetchFile-268 | ResponseHelper | Sending error response: File Y:?DFE_Build-dfe-component-build_current-core-8.0-18453944\init-svn-change-chec.0e1e1cac-7d79-11e9-bc46-0645b93d2238.log not found.
So, in this case, we can confirm that the culprit is in the server side. Deleting the workspace and recreating it or editing it to use some path like "c:/path/in/c/drive" may help.
