ATLAS Grid Information System

DDM blacklisting data
ATLAS SiteDDMEndpoint
Status
DeleteDELETE LANDELETE WANFetchPilot Log WritePilot ReadPilot WriteReadREAD LANREAD WANSubscriptiondefault SRMThird party copyUploadWriteWRITE LANWRITE WAN
CERN-PROD CERN-PROD_DERIVED manual manual
CERN-PROD_PPSTAPE manual manual manual manual manual manual
CERN-PROD_RAW manual manual
CERN-PROD_TRIG-DAQ QUOTASPACE QUOTASPACE
CERN-PROD_TZDISK manual manual
CSCS-LCG2 CSCS-LCG2_LOCALGROUPDISK DISKSPACE DISKSPACE
FZK-LCG2 FZK-LCG2_PERF-IDTRACKING QUOTASPACE QUOTASPACE
HK-LCG2 HK-LCG2_DATADISK AGIS
HK-LCG2_SCRATCHDISK AGIS
NDGF-T1 NDGF-T1_GROUPTAPE_PHYS-TOP QUOTASPACE QUOTASPACE
NERSC-PDSF NERSC_DATADISK manual manual manual manual manual manual
NIKHEF-ELPROD NIKHEF-ELPROD_LOCALGROUPDISK DISKSPACE DISKSPACE
NIKHEF-ELPROD_SCRATCHDISK manual manual manual manual manual
SMU_HPC SMU2_LOCALGROUPDISK manual manual manual manual manual manual
TOKYO-LCG2 TOKYO-LCG2_DET-MUON QUOTASPACE QUOTASPACE
TUDresden-ZIH TUDRESDEN-ZIH_LOCALGROUPDISK manual manual manual manual manual + manual +
UKI-LT2-RHUL UKI-LT2-RHUL_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-NORTHGRID-LIV-HEP UKI-NORTHGRID-LIV-HEP_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-NORTHGRID-SHEF-HEP UKI-NORTHGRID-SHEF-HEP_DATADISK manual manual
UKI-NORTHGRID-SHEF-HEP_LOCALGROUPDISK manual manual
Rules and Threshold values: /afs/cern.ch/atlas/GRID/AGIS/ddmblacklisting/config/ddmblacklist_config.py   Description

DDMBlacklisting JSON API
1. To retrieve data in JSON format, filter by fstate (final state) or permission (activity) values:
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=[ON|OFF]
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=[ON|OFF]&activity=[d,f,r,s,u,w]
Examples,
The equivalents of DQ2 blacklisting functions:
  • DQ2.listWhitelistedLocationsForAction("w")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=ON&activity=w&preset=ddmcompat | JSON example
  • DQ2.listBannedLocationsForAction("w")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=OFF&activity=w&preset=ddmcompat | JSON example
  • DQ2.listBannedLocationsByCondition("DISKSPACE")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&preset=ddmcompatbycondition | JSON example
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&preset=ddmcompatbycondition&provider=DISKSPACE | JSON example
2. Curl like API GET requests to update DDMBlacklisting data:
  • GET is accepted HTTP method to submit data
  • To update (manual) state of DDMEndpoints, the equivalent of DQ2 dq2-set-location-status command:
    1. accepted keys: ddmendpoint or site, activity, value, reason, [expiration]
    2. GET URI format:
    3. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_status/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=[dfrsuw]&value=[ON|OFF|AUTO]&reason=comment&expiration=2013-01-01T12:10:00
  •  
  • To blacklist whole site (set manual state for all ddmendpoints behind the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[dfrsuw]&value=[ON|OFF]&provider=[SRMPing,DISKSPACE,AGIS]&reason=comment&expiration=2013-07-07T12:00:00
  •  
  • To update autostate value of DDMEndpoint for specific condition (by various providers):
    1. accepted keys: provider, ddmendpoint, activity, value, reason, expiration
    2. GET URI format:
    3. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_probestatus/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=[dfrsuw]&value=[ON|OFF]&provider=[SRMPing,DISKSPACE,AGIS]&reason=comment&expiration=2013-07-07T12:00:00
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.

  • Example:
  • Set Manual status: excluding 2 ddmendpoints (DDM1,DDM2) for permission flags u,w and s:
  •   -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_status/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=uws&value=OFF&reason=temporary_excluded
3. To retrieve the history of DDMEndpoint status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&provider=[DISKSPACE|AGIS|SRMPing|manual]&activity=[d,f,r,s,u,w]
Examples,
The equivalents of DQ2 blacklisting functions:
  • DQ2.getStatusHistory(locations=['DDM1', 'DDM2'], activity=['u','w'])
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&preset=ddmcompat&show=manual&activity=rw&ddmendpoint[]=DDM1&ddmendpoint=DDM2 | JSON example
  • DQ2.getConditionHistory(locations=['DDM1', 'DDM2'], activity=['u','w'])
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&preset=ddmcompat&show=auto&activity=rw&ddmendpoint[]=DDM1&ddmendpoint=DDM2 | JSON example