FTS3DB
Frontend to FTS3 MySQL DB. Written using sqlalchemy
- class DIRAC.DataManagementSystem.DB.FTS3DB.FTS3DB(pool_size=15, url=None, parentLogger=None)
Bases:
object- class RequestDB
db holding requests
- __init__(pool_size=15, url=None, parentLogger=None)
c’tor
- Parameters
self – self reference
pool_size – size of the connection pool to the DB
- cancelNonExistingJob(operationID, ftsGUID)
Cancel an FTS3Job with the associated FTS3Files. This is to be used when the job is not found on the server when monitoring.
The status of the job and files will be ‘Canceled’. The error is specifying that the job is not found. The ftsGUID of the file is released as to be able to pick it up again
- Parameters
operationID – guess
ftsGUID – guess
- Returns
S_OK() if successful, S_ERROR otherwise
- createTables()
create tables
- deleteFinalOperations(limit=20, deleteDelay=180)
deletes operation in final state that are older than given time
- getActiveJobs(limit=20, lastMonitor=None, jobAssignmentTag='Assigned')
- Get the FTSJobs that are not in a final state, and are not assigned for monitoring
or has its operation being treated
- By assigning the job to the DB:
it cannot be monitored by another agent
the operation to which it belongs cannot be treated
- Parameters
limit – max number of Jobs to retrieve
lastMonitor – jobs monitored earlier than the given date
jobAssignmentTag – if not None, block the Job for other queries, and use it as a prefix for the value in the operation table
- Returns
list of FTS3Jobs
- getNonFinishedOperations(limit=20, operationAssignmentTag='Assigned')
Get all the non assigned FTS3Operations that are not yet finished, so either Active or Processed. An operation won’t be picked if it is already assigned, or one of its job is.
- Parameters
limit – max number of operations to retrieve
operationAssignmentTag – if not None, block the operations for other queries, and use it as a prefix for the value in the operation table
- Returns
list of Operations
- getOperation(operationID)
read request
This does not set the assignment flag
- Parameters
operationID – ID of the FTS3Operation
- getOperationsFromRMSOpID(rmsOpID)
Returns the FTS3Operations matching a given RMS OperationID
This does not set the assignment flag
- Parameters
rmsOpID – ID of the RMS Operation
- kickStuckJobs(limit=20, kickDelay=2)
- finds jobs that have not been updated for more than a given
time but are still assigned and resets the assignment
- kickStuckOperations(limit=20, kickDelay=2)
- finds operations that have not been updated for more than a given
time but are still assigned and resets the assignment
- persistOperation(operation)
- update or insert request into db
Also release the assignment tag
- Parameters
operation – FTS3Operation instance
- updateFileStatus(fileStatusDict, ftsGUID=None)
- Update the file ftsStatus and error
The update is only done if the file is not in a final state (To avoid bringing back to life a file by consuming MQ a posteriori)
TODO: maybe it should query first the status and filter the rows I want to update !
- Parameters
fileStatusDict – { fileID : { status , error, ftsGUID } }
ftsGUID – If specified, only update the rows where the ftsGUID matches this value. This avoids two jobs handling the same file one after another to step on each other foot. Note that for the moment it is an optional parameter, but it may turn mandatory soon.
- updateJobStatus(jobStatusDict)
- Update the job Status and error
The update is only done if the job is not in a final state The assignment flag is released
- Parameters
jobStatusDict – { jobID : { status , error, completeness } }