Electroserver resource utilization logging

[theworldinunion]

In ES5 Admin, there is a "Server Monitoring"->"Status" screen that tells you how much CPU, memory, throughput, and how many connected users that Electroserver is going through at this moment.

This is cool, but due to 2 large downsides, in a serious production environment this screen is unfortunately impractical and doesn't work :
1) You'd have to hire someone at a cost of a few thousand dollars a month to stare at ES5 admin the whole day, and alert people immediately when they see any peaks.
2) You can't capture a historical chart of resource usage, so future resource planning is impossible.

We'd like to automatically send these usage stats to a resource monitoring tool like munin. Can you tell us which functions to call, so that we can grab these statistics like ES5Admin is doing?

Thanks.

[tcarr]

The statistics that appear in the ES Admin are all available using JMX. If that article and the links from it doesn't give you enough information, let me know and I'll ask our JMX expert for more details on how this is done. I've used the MBeans a few times from a plugin, but haven't ever done the JMX monitoring myself.

edit: have you looked at the ES Admin's Server Monitoring - Reporting screen? That will give you historical graphs.

edit 2: Here's a note from the manual's migration roadmap that might assist if you want to use MBeans.

MBeans were changed drastically, and in some cases the information is more easily accessed now using ElectroServerApi. For example, in ES4 the only way to get the number of connected users was to use the MBeanServer, but in ES5 you can use getApi().getUserCount(). The new MBean for number of connected users would replace com.electrotank.electroserver4:type=ConnectedUsers and Count with com.electrotank.electroserver:type=ElectroServer.S tatistics and NumberOfUsers, respectively, if you need to continue using an MBean.

[theworldinunion]

As always, thanks for quick reply Teresa!
Just to clarify, is there any way to get (1) CPU, (2) memory, (3) throughput stats via getApi(), or would MBeans be the only way of getting this?
Any quick pointers are appreciated, I'd like to make extracting these stats as painless as possible (after all, there are more fun things to play with in Electroserver ;-)

[tcarr]

These three are not exposed via the api. You would need to use MBeans or JMX or JConsole or something. See this url for info on memory and cpu (not ES5 specific). I'd have to ask Jason about the throughput stats - let me know if you want me to do that.

[theworldinunion]

Would be great if you can help me ask Jason. if possible I'd like to pull stats with the same methodology as ESAdmin for consistency, so just some pointers to the right direction on how you guys (who understands the internals of ES) pulled throughput stats from ES5 into ESAdmin would suffice.

[tcarr]

See if this info is enough for you. I *think* I am guessing which part of the source code determines the MBean attributes. Try using "com.electrotank.electroserver:type=ElectroServer. Statistics" and any of the following:
SystemLoadAverage
VmMemoryFree
VmMemoryTotal
VmMemoryMax
NumberOfUsers
MessagesInTotal
MessagesOutTotal
BytesInTotal
BytesOutTotal

If this works for you, let me know and I'll add this info to the manual for the next time that somebody asks about it.

ES Admin gets its info using JMX, which gets it from the MBeans.

Another MBean we have is "com.electrotank.electroserver:type=ElectroSer ver" that gives information on rooms, zones, users, etc.

edit: remove any space in the quoted MBean name. The forum adds spaces to long strings so that they wrap.

[theworldinunion]

Hi Teresa,

Struggled with this for days, but to no avail. I can't find any relevant documentation as to the MBean attributes that you are using.

in my /etc/collectd.conf ( see http://collectd.org/wiki/index.php/Plugin:GenericJMX ) :

<Plugin "java">
  JVMARG "-Djava.class.path=/usr/share/collectd/java/collectd-api.jar:/usr/share/collectd/java/generic-jmx.jar"
  LoadPlugin "org.collectd.java.GenericJMX"

  <Plugin "GenericJMX">
    # Memory usage by memory pool.  
    <MBean "messages">
      ObjectName "com.electrotank.electroserver:type=ElectroServer"
      <Value>
        Type "gauge"
        InstancePrefix "Statistics"
        #InstanceFrom ""
        Table false
        Attribute "MessagesInTotal"
      </Value>
    </MBean>

    <Connection>
       Host "localhost"
       ServiceURL "service:jmx:rmi:///jndi/rmi://localhost:9999/jmxrmi"
       Collect "messages"
    </Connection>

  </Plugin>
</Plugin>

in syslog :

collectd[1547]: GenericJMXConfValue.query: getAttribute failed: javax.management.AttributeNotFoundException: No such attribute: MessagesInTotal
collectd[1547]: GenericJMXConfValue.query: Querying attribute MessagesInTotal failed.

In the link you provided : http://www.electrotank.com/docs/es5/...x.html?jmx.htm , I quote : "You can also access the MBeans from plugins. You can follow the patterns here." The "here" link points to just a ES5Admin screenshot with no pattern documentation.

Can you please check with your JMX expert with regards to what I should be putting into collectd.conf as "ObjectName", "InstancePrefix" and "Attribute" ? I have tried multiple variations in the absence of documentation.

As mentioned, ES5Admin is cool, but a serious production environment will want to have alerts automated with centralized monitoring. If you can provide me with enough help for me to get this working, I am happy to post full instructions on how to automate the collection of Electroserver statistics using collectd , so that you can put it in your manual. This will make Electroserver a lot more attractive to serious users.

[tcarr]

I'll be back at my desk on Wednesday. I'm making a ticket so that this request doesn't get lost.

[jvonnieda]

ObjectName is "com.electrotank.electroserver:type=ElectroServer. Statistics"
There is no InstancePrefix
The rest of your values look correct.

I find the easiest way to figure out these values are to use jconsole to browse to an MBean that you are interested in. jconsole will show all the necessary naming values and then you can easily pick and choose the attributes you are interested in.

[tcarr]

In the link you provided : http://www.electrotank.com/docs/es5/...x.html?jmx.htm , I quote : "You can also access the MBeans from plugins. You can follow the patterns here." The "here" link points to just a ES5Admin screenshot with no pattern documentation.

I found the bug in the manual that was giving the wrong link for "here". It should be this one instead:
http://java.sun.com/javase/6/docs/te...t/mxbeans.html