| | |
- object
-
- MassRetriever
class MassRetriever(object) |
| |
Table for retrieving value sets from multiple agents
The object wraps a single query, it cannot be shared among
multiple queries!
|
| |
Methods defined here:
- __call__(self, oids=(), tables=(), iterDelay=0.0050000000000000001, *arguments, **named)
- Do mass retrieval of oids and tables from our proxies
oids -- individual (get) oids for each Agent
tables -- multi-value (getTable) oids for each Agent
** named -- passed to proxy object's get and getTable
methods, so retryCount and timeout can be specified
returns a DeferredList with the results of all queries
raises ValueError if no proxies, or neither oids or tables
The DeferredList's callback recieves a set mapping:
{ (ip,port) : valueSet }
where valueSet is a set mapping:
{ queriedOID : dataValues }
where dataValues is as returned from get/getTable on the
proxy object.
Register callbacks on the DeferredList to recieve a set of
results in (success,result) format, where success is a boolean
and result is either the result or an ErrBack object.
- __init__(self, proxies)
- Initialise the retriever with client AgentProxies
proxies -- sequence of properties to query in batches
- handleSingleError(self, err, oids, proxy)
- Handle single-agent error by making result[key][oid] None
err -- handle complete failure of a single proxy to retrieve
values.
oids -- the oids being queried
proxy -- the proxy being queried
Values are added to the (ip,port) dictionary for the proxy
with value None iff the root OID does not already exist in
the dictionary.
returns None
- integrateSingleResult(self, value, proxy)
- Integrate single agent-query result into mega-result
value -- dictionary value as returned from get/getTable
of the proxy object
proxy -- the proxy which was used to retrieve the value
returns value, if a pre-existing dictionary for proxy
(determined by ip,port) exists, updates that dictionary
with the new values, otherwise creates a new dictionary
and updates that.
Note: Implication here is that, if two proxies for the
same ip and port are being scanned, their results
will comingle, and potentially overwrite one another.
- printStats(self)
- Print heuristic stats for the retriever
- returnFinal(self, dataList)
- Handle final defer callback from completion of all partialDefers
When all proxies have completed (successfully or not), we callback
our finalDefer to let our client know about our status. finalDefer
gets a pointer to self.result, and the method also returns
self.result, just for good measure.
- singleProxy(self, proxy, oids, tables)
- Add configured callbacks for a given agent proxy
proxy -- the proxy to be queried
oids -- single oids to be queried (get)
tables -- multi-value oids to be queried (getTable)
- smallBatch(self, oids, tables, index=0, iterDelay=0.01)
- Do single-proxy batches iteratively
This is the most robust algorithm I've come up with so far,
it has the rather poor feature of limiting queries to
1/iterDelay per sec at maximum, but it avoids the annoying problems
which crop up using nonBatch or proxyBatch implementations.
Practically speaking, systems seem quite capable of servicing
a few thousand requests/sec if there's enough processing power,
but twisted's strange performance characteristics make it
difficult to write a beast that works to that level AFAICS.
Data and other attributes defined here:
- __dict__ = <dictproxy object>
- dictionary for instance variables (if defined)
- __weakref__ = <attribute '__weakref__' of 'MassRetriever' objects>
- list of weak references to the object (if defined)
- errorCount = 0
- successCount = 0
| |