ATLAS Grid Information System

DDM blacklisting data
ATLAS SiteDDMEndpoint
Status
DeleteDELETE LANDELETE WANFetchPilot Log WritePilot ReadPilot WriteReadREAD LANREAD WANSubscriptiondefault SRMThird party copyUploadWriteWRITE LANWRITE WAN
BNL-ATLAS BNL-OSG2_PHYS-HI QUOTASPACE QUOTASPACE
CERN-EXTENSION CERN-EXTENSION_DATADISK manual manual manual manual manual manual
CERN-EXTENSION_SCRATCHDISK manual manual manual manual manual manual
CERN-PROD CERN-PROD_PERF-FLAVTAG QUOTASPACE QUOTASPACE
CERN-PROD_TRIG-DAQ QUOTASPACE QUOTASPACE
CERN-PROD_TZDISK manual manual
FZK-LCG2 FZK-LCG2_PERF-IDTRACKING QUOTASPACE QUOTASPACE
INFN-ROMA3 INFN-ROMA3_LOCALGROUPDISK DISKSPACE DISKSPACE
NDGF-T1 NDGF-T1_DATADISK-DATAFJORD manual manual manual manual manual manual
NDGF-T1_GROUPTAPE_PHYS-TOP QUOTASPACE QUOTASPACE
NERSC-PDSF NERSC_DATADISK manual manual manual manual manual manual
SE-SNIC-T2 SE-SNIC-T2_UPPSALA_LOCALGROUPDISK DISKSPACE DISKSPACE
SMU_HPC SMU2_LOCALGROUPDISK manual manual manual manual manual manual
TOKYO-LCG2 TOKYO-LCG2_DET-MUON QUOTASPACE QUOTASPACE
TUDresden-ZIH TUDRESDEN-ZIH_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-NORTHGRID-LIV-HEP UKI-NORTHGRID-LIV-HEP_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-SOUTHGRID-OX-HEP UKI-SOUTHGRID-OX-HEP_LOCALGROUPDISK DISKSPACE DISKSPACE
Rules and Threshold values: /afs/cern.ch/atlas/GRID/AGIS/ddmblacklisting/config/ddmblacklist_config.py   Description

DDMBlacklisting JSON API
1. To retrieve data in JSON format, filter by fstate (final state) or permission (activity) values:
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=[ON|OFF]
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=[ON|OFF]&activity=[d,f,r,s,u,w]
Examples,
The equivalents of DQ2 blacklisting functions:
  • DQ2.listWhitelistedLocationsForAction("w")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=ON&activity=w&preset=ddmcompat | JSON example
  • DQ2.listBannedLocationsForAction("w")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=OFF&activity=w&preset=ddmcompat | JSON example
  • DQ2.listBannedLocationsByCondition("DISKSPACE")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&preset=ddmcompatbycondition | JSON example
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&preset=ddmcompatbycondition&provider=DISKSPACE | JSON example
2. Curl like API GET requests to update DDMBlacklisting data:
  • GET is accepted HTTP method to submit data
  • To update (manual) state of DDMEndpoints, the equivalent of DQ2 dq2-set-location-status command:
    1. accepted keys: ddmendpoint or site, activity, value, reason, [expiration]
    2. GET URI format:
    3. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_status/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=[dfrsuw]&value=[ON|OFF|AUTO]&reason=comment&expiration=2013-01-01T12:10:00
  •  
  • To blacklist whole site (set manual state for all ddmendpoints behind the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[dfrsuw]&value=[ON|OFF]&provider=[SRMPing,DISKSPACE,AGIS]&reason=comment&expiration=2013-07-07T12:00:00
  •  
  • To update autostate value of DDMEndpoint for specific condition (by various providers):
    1. accepted keys: provider, ddmendpoint, activity, value, reason, expiration
    2. GET URI format:
    3. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_probestatus/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=[dfrsuw]&value=[ON|OFF]&provider=[SRMPing,DISKSPACE,AGIS]&reason=comment&expiration=2013-07-07T12:00:00
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.

  • Example:
  • Set Manual status: excluding 2 ddmendpoints (DDM1,DDM2) for permission flags u,w and s:
  •   -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_status/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=uws&value=OFF&reason=temporary_excluded
3. To retrieve the history of DDMEndpoint status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&provider=[DISKSPACE|AGIS|SRMPing|manual]&activity=[d,f,r,s,u,w]
Examples,
The equivalents of DQ2 blacklisting functions:
  • DQ2.getStatusHistory(locations=['DDM1', 'DDM2'], activity=['u','w'])
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&preset=ddmcompat&show=manual&activity=rw&ddmendpoint[]=DDM1&ddmendpoint=DDM2 | JSON example
  • DQ2.getConditionHistory(locations=['DDM1', 'DDM2'], activity=['u','w'])
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&preset=ddmcompat&show=auto&activity=rw&ddmendpoint[]=DDM1&ddmendpoint=DDM2 | JSON example