Troubleshooting

First-use issues

IVP

Performing DWAgent IVP is a very effective technique to identify first-use problems as early as possible, especially after the configuration has been altered.
A failure in IVP is usually qualified by a verbose error in one or more logs in <DWAHOME>\logs, which can help in identifying the root cause of the problem.
You perform IVP by running this command:
dwagent.sh -ivp

INVESTIGATING MISSING PREREQUISITES, NOCLASSDEFFOUND EXCEPTIONS

DWAgent is not a self-contained product. Depending on the emitters and proxies that are actually used in a specific installation, one or more prerequisite products must be available at runtime.
All these products must be correctly referenced in the initial configuration of each component, as previously explained in this manual.
A common symptom of an incorrectly referenced product prerequisite is a NoClassDefFound exception in DWAgent product logs that usually translates into a problem (e.g., a spelling error or a missing variable definition) in the environment read by DWAgent at runtime.
In a complex production scenario, determining the actual environment variables available to the DWAgent at runtime might not be easy.
A very effective technique to cope with this kind of problem is to take a user dump of a running DWAgent and inspect the section where an integral textual dump of the actual environment observed by DWAgent is reported.
For example, you can take a user dump of a live instance of DWAgent by running this command:
dwagent.sh -usrdump

FAILURE TO RUN DWAGENT AS A WINDOWS SERVICE

In rare circumstances (depending on the combination of the Windows version being used and/or JRE versions and brands being used) a correctly registered Windows Service DWAgent may fail upon start-up reporting the following error messages in <DWAHOME>\logs\dwagentservice.log:

[2020-02-16 10:16:56] [174 javajni.c] [error] The specified module could not be found.
[2020-02-16 10:16:56] [994 prunsrv.c] [error] Failed creating java c:\program files\java\jre6\bin\server

The root cause of this problem is a failure to load the correct instance of the Microsoft C Runtime Library dll, i.e. the one redistributed in your JRE bin directory.
To work around this problem, copy any msvcr*.dll (e.g. msvcr71.dll) from your JRE bin directory to <DWAHOME>\bin\runtimeW64 directory.

Generic runtime problems

INVESTIGATING A “NO EVENTS SHOWN ON DATA WATCHER” SYMPTOM USING DWAGENT STATISTICS

In an apparently well configured system, the root cause of a “no events shown on Data Watcher” symptom might be:

• Events not being detected by the relevant DWA Emitter on DWAgent
• Events not being forwarded by DWA Proxy
• Events being discarded by Data Watcher

The DWAgent related symptoms can be easily investigated by inspecting DWAgent statistics printed on the console using the following command:

dwagent.sh -stats

For instance, if events are not being detected by an XFERLOG DWAEmitter this will result in Emitter\XFERLOG\inputTotal=0 or Emitter\XFERLOG\inputLastTimestamp thief in the distant past.

If this is not the case it might be that all events detected are discarded by DWAEmitter, in which case you would observe different values for Emitter\XFERLOG\inputTotal and Emitter\XFERLOG\outputTotal.

DWAEmitters discarding events may be normal (some events are not relevant). Any anomalies in this area might be detected by checking Emitter\XFERLOG\inputTotalError and Emitter\XFERLOG\inputLastErrorTimestamp.

The following is a snippet of available DWAEmitter statistics:

Emitter\XFERLOG\inputTotal=25
Emitter\XFERLOG\inputLastTimestamp=20100521100459727
Emitter\XFERLOG\inputLastCursorTimestamp=20100521100459000
Emitter\XFERLOG\inputTotalError=0
Emitter\XFERLOG\inputLastErrorTimestamp=-1
Emitter\XFERLOG\outputTotal=25 Emitter\XFERLOG\outputLastTimestamp=20100521100459730
Emitter\XFERLOG\outputTotalError=0
Emitter\XFERLOG\outputLastErrorTimestamp=-1

For instance, if a DWCommunicator problem prevents events from being sent, this will result in Communicator\ARTEMIS\sendTotal=0 or Communicator\ARTEMIS\sendLastTimestamp in the distant past and a recent Communicator\ARTEMIS\sendLastErrorTimestamp.

The following is a snippet of available DWCommunicator statistics:

Communicator\ARTEMIS\sendTotal=91
Communicator\ARTEMIS\sendLastTimestamp=20100521100842913
Communicator\ARTEMIS\sendBacklog=0
Communicator\ARTEMIS\sendTotalError=4612
Communicator\ARTEMIS\sendLastErrorTimestamp=20100521070246788

TURNING RUNTIME TRACING ON AND OFF

As part of problem determination or when specifically instructed to do so by PRIMEUR Contact Center, you might need to turn on runtime tracing of a running DWAgent.
On Unix machines this can be done by running the following command:

dwagent.sh –trace verbose

This will typically be followed by running the failing use case to reproduce the problem being investigated and then the following command to restore the default trace level:

dwagent.sh –trace default

Each time the trace level is changed, an automatic user dump is taken. Traces and dumps can subsequently be collected for problem determination when reporting the problem to PRIMEUR Contact Center.

See the System Administration section for more details on trace management and user dumps.

TURNING EARLY TRACING ON AND OFF

For some types of failures occurring very soon after DWAgent startup, it is not feasible to manage traces as described in the previous section.

To deal with these scenarios, you can turn on early tracing by manually editing <DWAHOME>\config\log4j.properties and resetting one or more severity related properties from INFO to DEBUG.

[…]
log4j.category.com.primeur.agentengine.emitter=DEBUG, DWAGENT
log4j.category.com.primeur.agentengine.communicator=DEBUG, DWAGENT
[…]

Hint!

You can find more information about the log4j java logging subsystem at this Apache Log4j link.