tag:blogger.com,1999:blog-1978652979840829013.post3859091744433048517..comments2024-02-13T07:05:41.069-08:00Comments on sFlow: TomcatPeterhttp://www.blogger.com/profile/00856599914190257147noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-1978652979840829013.post-59915655197102088262015-08-19T17:28:44.010-07:002015-08-19T17:28:44.010-07:00I added it to startup.sh :
export CATALINA_OPTS=&q...I added it to startup.sh :<br />export CATALINA_OPTS="$CATALINA_OPTS -Dsflow.dsindex=1234 -Dsflow.hostname=app_1"<br /><br />Now SflowValve,jar is working.Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-46156593130765640292015-08-19T12:12:12.680-07:002015-08-19T12:12:12.680-07:00I'm very limited in setting up Tomcat environm...I'm very limited in setting up Tomcat environments. I can only adjust monitoring parts. Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-1254272648078038522015-08-19T10:58:00.033-07:002015-08-19T10:58:00.033-07:00You can set the sflow.dsindex system property in t...You can set the sflow.dsindex system property in the Tomcat setenv.sh script, add a line like the following:<br /><br />export CATALINA_OPTS="$CATALINA_OPTS -Dsflow.dsindex=1234 -Dsflow.hostname=app_1"<br /><br />setenv.sh is in the Tomcat bin directory.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-78783272979189698062015-08-19T10:40:00.809-07:002015-08-19T10:40:00.809-07:00In the case where sflow.dsindex is not set as a Sy...In the case where sflow.dsindex is not set as a System property, it will be set in sampleRequest(), <br /><br />if(dsIndex == -1) dsIndex = request.getLocalPort();<br /><br />Are you generating any http requests to Tomcat? You can adjust the sampling rate by setting the parameter sampling.http in the hsflowd.conf file (you need to restart hsflowd after making the change).Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-20597621886708424712015-08-19T10:20:28.148-07:002015-08-19T10:20:28.148-07:00This is a piece of SflowValve.java:
public sta...This is a piece of SflowValve.java:<br /><br /> public static int dsIndex() {<br /> int dsIndex = -1;<br /> String indexStr = System.getProperty("sflow.dsindex");<br /> if(indexStr != null) {<br /> try { dsIndex = Integer.parseInt(indexStr); }<br /> catch(NumberFormatException e) {<br /> }<br /> }<br /> return dsIndex;<br /> }<br /><br />It's called from here:<br /><br /> private void pollCounters(long now) {<br /><br /> if(agentAddress == null) return;<br /> if(dsIndex == -1) return;<br /> ...<br /><br />I see that it always get (dsIndex ==-1 )=> return on this place.<br />I tried to comment this out and now it sends HTTP and JVM metrics.<br /><br />Do I use write source code?Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-1620205075324125892015-08-18T18:29:07.788-07:002015-08-18T18:29:07.788-07:00Do the settings in your /etc/hsflowd.auto file loo...Do the settings in your /etc/hsflowd.auto file look correct (see <a href="http://blog.sflow.com/2012/01/host-sflow-distributed-agent.html" rel="nofollow">Host sFlow distributed agent</a>?Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-61019069514767912612015-08-18T18:27:04.381-07:002015-08-18T18:27:04.381-07:00You don't need to set the sflow.dsindex proper...You don't need to set the sflow.dsindex property.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-44733140827471609182015-08-18T18:20:58.191-07:002015-08-18T18:20:58.191-07:00You must also be running Host sFlow? The current i...You must also be running <a href="http://blog.sflow.com/2010/10/installing-host-sflow-on-linux-server.html" rel="nofollow">Host sFlow</a>? The current implementation relies on the Host sFlow configuration file - see <a href="http://blog.sflow.com/2012/01/host-sflow-distributed-agent.html" rel="nofollow">Host sFlow distributed agent</a>. Currently only Linux based platforms are support - there is no integration between Host sFlow and the Tomcat Valve on Windows.<br /><br />How are you verifying whether data is being sent? Have you tried <a href="http://blog.sflow.com/2011/12/sflowtool.html" rel="nofollow">sflowtool</a>?Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-847608282690424832015-08-18T17:08:33.919-07:002015-08-18T17:08:33.919-07:00I don't see Request.getBytesRead() exception ...I don't see Request.getBytesRead() exception anymore, but I don't get any data.<br />Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-11031665704612074282015-08-18T17:06:09.324-07:002015-08-18T17:06:09.324-07:00Hi Peter,
I added logging to SFlowValve.java
Now I...Hi Peter,<br />I added logging to SFlowValve.java<br />Now I see that System.getProperty("sflow.dsindex") call always returns "null".<br />I work on Linux version 2.6.39-400.214.4.el6uek.x86_64. <br />Hsflowd is running normally. Any help?Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-86185700019815909782015-08-12T08:37:27.572-07:002015-08-12T08:37:27.572-07:00When you say re-compiling didn't help, what do...When you say re-compiling didn't help, what do you mean? Are you still getting an exception from the Request.getBytesRead() method? <br /><br />Are you running on Windows? The current implementation relies on the Host sFlow configuration file and only works on Linux based systems.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-62805184409823159122015-08-05T16:46:05.250-07:002015-08-05T16:46:05.250-07:00I still didn't get any answer. I only know tha...I still didn't get any answer. I only know that HTTP Connector Coyote was changed in 7.0.60.<br />I tried to recompile SflowValve.java for Tomcat-7.0.61 - it didn't help. Any ideas? Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-38256066324612667022015-07-23T09:47:58.020-07:002015-07-23T09:47:58.020-07:00We updated Tomcat from 7.0.10 to 7.0.61. In previo...We updated Tomcat from 7.0.10 to 7.0.61. In previous version we collected Tomcat metrics using Sflow_Valve.jar module. It looks like Coyote Connector can't call Request.getBytesRead() method.<br />Did anybody have this issue?Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-34454713009452233832015-07-21T13:31:41.836-07:002015-07-21T13:31:41.836-07:00We updated Tomcat from 7.0.10 to 7.0.61. In previo...We updated Tomcat from 7.0.10 to 7.0.61. In previous version we collected Tomact mwtrics usin Sflow_Valve.jar module. New version gives following errors:<br />org.apache.coyote.http11.AbstractHttp11Processor process<br />SEVERE: Error processing request<br />java.lang.NoSuchMethodError: org.apache.coyote.Request.getBytesRead()I<br /> at com.sflow.catalina.SFlowValve.xdrFlowSample(SFlowValve.java:341)<br /> at com.sflow.catalina.SFlowValve.sampleRequest(SFlowValve.java:438)<br /> at com.sflow.catalina.SFlowValve.invoke(SFlowValve.java:320)<br /> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)<br /> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)<br /> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:423)<br /> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1079)<br /> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:620)<br /> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)<br /> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)<br /> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)<br /> at java.lang.Thread.run(Unknown Source)<br /><br />Any help will be very appreciated.Anonymoushttps://www.blogger.com/profile/03775220930331770028noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-56405095889077703042014-06-10T12:31:02.594-07:002014-06-10T12:31:02.594-07:00we are running hsflowd but the hsflowd.auto is not...we are running hsflowd but the hsflowd.auto is not in /etc/hsflowd.auto. Can we tell sflowValue where to find hsflowd.autogmaxwellhttps://www.blogger.com/profile/16431240080570373530noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-53197932453792747952014-06-10T10:53:44.526-07:002014-06-10T10:53:44.526-07:00Thanks. I fixed the typo in the instructions.Thanks. I fixed the typo in the instructions.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-77065445832348996952014-06-10T10:42:11.337-07:002014-06-10T10:42:11.337-07:00The Tomcat valve is intended to be used with a Hos...The Tomcat valve is intended to be used with a Host sFlow agent (which will write the hsflowd.auto file) - see <a href="http://blog.sflow.com/2012/01/host-sflow-distributed-agent.html" rel="nofollow">Host sFlow distributed agent</a>. sFlow measurements from the network, host, virtual switches, VMs and applications are part of an integrated system that links the performance of the layers together - see <a href="http://blog.sflow.com/2010/08/sflow-host-structures.html" rel="nofollow">sFlow Host Structures</a>. An sFlow collector will expect to see host performance statistics from hosts exporting HTTP statistics.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-19002798271124254802014-06-10T10:06:34.882-07:002014-06-10T10:06:34.882-07:00We have hsflowd running in user space, without per...We have hsflowd running in user space, without permission to add / create the /etc/hsflowd.auto<br />so our hsflowd.auto is in /home/(user name)/run/hsflowd.auto. Can we add a -Dhsflowhome= to use a different location for hsflowd.auto.gmaxwellhttps://www.blogger.com/profile/16431240080570373530noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-19158077901140209852014-06-10T09:51:28.641-07:002014-06-10T09:51:28.641-07:00it is a typo classname should be className
<...it is a typo classname should be className<br /> <-- wrong<br /> <-- correctgmaxwellhttps://www.blogger.com/profile/16431240080570373530noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-50036719453138996382014-01-07T10:54:19.903-08:002014-01-07T10:54:19.903-08:00I don't believe there is a way to do what you ...I don't believe there is a way to do what you are asking with Ganglia since Ganglia assumes there may be more than one web server per hostname and it falls back on the index (which would normally be the port that the web server is listening on and a useful way of labeling the instance).<br /><br />Have you looked at the HTTP transaction data being exported via sFlow (URL, host, user agent, response time, status code etc)? Ganglia isn't able to report on this data, but you might want to try <a href="http://blog.sflow.com/2013/11/metric-export-to-graphite.html" rel="nofollow">sFlow-RT</a> or <a href="http://blog.sflow.com/2012/10/sflowtrend-adds-web-server-monitoring.html" rel="nofollow">sFlowTrend</a>.<br /><br />If you want to run additional tools in parallel with Ganglia, you can configure the Host sFlow agents to send to multiple destinations, or use <a href="http://blog.sflow.com/2012/01/forwarding-using-sflowtool.html" rel="nofollow">sflowtool/tcpdump</a> to replicate and forward the sFlow from your gmond collector.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-81379342133338349442014-01-07T09:55:12.714-08:002014-01-07T09:55:12.714-08:00Thanks for getting back to me, we're using Gan...Thanks for getting back to me, we're using Ganglia as the reporting software. The way they have implemented the JVM here is for each application they develop they will create a new separate JVM bound to port 8080 and a new virtual IP address assigned to the same physical host OS/ adapter. So as an example we basically we have one server running App1 with alias ip of X.X.X.01, App2 with alias IP of X.X.X.2, App3 with alias IP of X.X.X.3 and so on. All of these alias IP's are on the same physical host OS/adapter. I'm not sure why they designed the JVM this way. <br />Here is what I see when I connect to the Ganglia gmond receiving port for application Moblie with the following set in the env:<br /><br />-Dsflow.dsindex=65539 -Dsflow.hostname=Moblie"<br /><br />HTTP metrics show with the number:<br /><br /><br />JVM metrics show with the name (in our case vitural IP Hostname):<br /><br /><br />Is there anyway to get http metric name to show as the jvm metric name?Anonymoushttps://www.blogger.com/profile/14481314279900534889noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-58032862035164004832014-01-07T08:43:38.870-08:002014-01-07T08:43:38.870-08:00What software are you using to analyze and display...What software are you using to analyze and display the sFlow data? In general, one can't assume a one to one correspondence between application instance and hostname since there can be multiple web server instances per physical / virtual machine. However, the containment hierarchy is explicit in the sFlow data model and it should be possible to query to find the hostname of the containing virtual machine and include this in reports.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-9097719925523136342014-01-07T08:22:40.044-08:002014-01-07T08:22:40.044-08:00Thank you again Peter, that worked perfectly. All ...Thank you again Peter, that worked perfectly. All of the JVM's we have running now show up with the hostname that I specify so the developers know which application is theirs. Hey when I look at the http metrics they are showing up with a name of the -Dsflow.dsindex number that I specified and not the Dsflow.hostname. The JVM metrics all show up with the Dsflow.hostname though. Is there a was to get the http metrics to use the hostname as well?Anonymoushttps://www.blogger.com/profile/14481314279900534889noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-74814231061087732572014-01-06T20:52:41.733-08:002014-01-06T20:52:41.733-08:00I made the changes to the tomcat6 version and rele...I made the changes to the tomcat6 version and released 0.6.2 as a download.Peterhttps://www.blogger.com/profile/00856599914190257147noreply@blogger.comtag:blogger.com,1999:blog-1978652979840829013.post-34748791823142452812014-01-06T19:41:37.245-08:002014-01-06T19:41:37.245-08:00Thanks Peter! That's exactly what I was lookin...Thanks Peter! That's exactly what I was looking for. Is there anyway that this can be created for Tomcat6 like the other branches? sflowvalve.jar.tomcat6. Again, thanks again for creating this.Anonymoushttps://www.blogger.com/profile/14481314279900534889noreply@blogger.com