ATLAS Grid Information System

DDM blacklisting data
ATLAS SiteDDMEndpoint
Status
DeleteDELETE LANDELETE WANFetchPilot Log WritePilot ReadPilot WriteReadREAD LANREAD WANSubscriptiondefault SRMThird party copyUploadWriteWRITE LANWRITE WAN
BEIJING-LCG2 BEIJING-LCG2_SCRATCHDISK manual manual
CERN-PROD CERN-PROD_DATADISK_DATALAKES_PPS manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_PNPI_TESTA manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_PNPI_TESTB manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_PNPI_TESTC manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_PNPI_TESTD manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_HYBRID_TESTA manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_HYBRID_TESTB manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_HYBRID_TESTC manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_HYBRID_TESTD manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_RAIN6_TESTA manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_RAIN6_TESTB manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_RAIN6_TESTC manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_RAIN6_TESTD manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_TESTA manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_TESTB manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_TESTC manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_RU_TESTD manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_TESTA manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_TESTB manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_TESTC manual manual manual manual manual manual
CERN-PROD_DATADISK_DATALAKES_PPS_TESTD manual manual manual manual manual manual
CERN-PROD_DERIVED manual manual
CERN-PROD_PPSTAPE manual manual manual
CERN-PROD_RAW manual manual
CERN-PROD_TRIG-DAQ QUOTASPACE QUOTASPACE
CERN-PROD_TZDISK manual manual
CSCS-LCG2 CSCS-LCG2_LOCALGROUPDISK DISKSPACE DISKSPACE
FZK-LCG2 FZK-LCG2_PERF-IDTRACKING QUOTASPACE QUOTASPACE
HK-LCG2 HK-LCG2_DATADISK AGIS
HK-LCG2_SCRATCHDISK AGIS
IL-TAU-HEP IL-TAU-HEP_DATADISK manual manual
IL-TAU-HEP_SCRATCHDISK manual manual
NDGF-T1 NDGF-T1_GROUPTAPE_PHYS-TOP QUOTASPACE QUOTASPACE
NERSC-PDSF NERSC_DATADISK manual manual manual manual manual manual
NIKHEF-ELPROD NIKHEF-ELPROD_LOCALGROUPDISK DISKSPACE DISKSPACE
NIKHEF-ELPROD_SCRATCHDISK manual manual manual manual manual
SMU_HPC SMU2_LOCALGROUPDISK manual manual manual manual manual manual
TOKYO-LCG2 TOKYO-LCG2_DET-MUON QUOTASPACE QUOTASPACE
TUDresden-ZIH TUDRESDEN-ZIH_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-NORTHGRID-LIV-HEP UKI-NORTHGRID-LIV-HEP_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-NORTHGRID-SHEF-HEP UKI-NORTHGRID-SHEF-HEP_DATADISK manual + manual + manual + manual + manual +
UKI-NORTHGRID-SHEF-HEP_LOCALGROUPDISK AGIS AGIS AGIS manual + manual +
UKI-SCOTGRID-ECDF UKI-SCOTGRID-ECDF_LOCALGROUPDISK DISKSPACE DISKSPACE
UKI-SCOTGRID-ECDF-RDF UKI-SCOTGRID-ECDF-RDF_DATADISK manual manual
UKI-SCOTGRID-ECDF-RDF_SCRATCHDISK manual + manual +
WEIZMANN-LCG2 WEIZMANN-LCG2_DATADISK manual manual
WEIZMANN-LCG2_SCRATCHDISK manual manual
Rules and Threshold values: /afs/cern.ch/atlas/GRID/AGIS/ddmblacklisting/config/ddmblacklist_config.py   Description

DDMBlacklisting JSON API
1. To retrieve data in JSON format, filter by fstate (final state) or permission (activity) values:
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=[ON|OFF]
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=[ON|OFF]&activity=[d,f,r,s,u,w]
Examples,
The equivalents of DQ2 blacklisting functions:
  • DQ2.listWhitelistedLocationsForAction("w")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=ON&activity=w&preset=ddmcompat | JSON example
  • DQ2.listBannedLocationsForAction("w")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&fstate=OFF&activity=w&preset=ddmcompat | JSON example
  • DQ2.listBannedLocationsByCondition("DISKSPACE")
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&preset=ddmcompatbycondition | JSON example
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/list/?json&preset=ddmcompatbycondition&provider=DISKSPACE | JSON example
2. Curl like API GET requests to update DDMBlacklisting data:
  • GET is accepted HTTP method to submit data
  • To update (manual) state of DDMEndpoints, the equivalent of DQ2 dq2-set-location-status command:
    1. accepted keys: ddmendpoint or site, activity, value, reason, [expiration]
    2. GET URI format:
    3. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_status/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=[dfrsuw]&value=[ON|OFF|AUTO]&reason=comment&expiration=2013-01-01T12:10:00
  •  
  • To blacklist whole site (set manual state for all ddmendpoints behind the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[dfrsuw]&value=[ON|OFF]&provider=[SRMPing,DISKSPACE,AGIS]&reason=comment&expiration=2013-07-07T12:00:00
  •  
  • To update autostate value of DDMEndpoint for specific condition (by various providers):
    1. accepted keys: provider, ddmendpoint, activity, value, reason, expiration
    2. GET URI format:
    3. -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_probestatus/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=[dfrsuw]&value=[ON|OFF]&provider=[SRMPing,DISKSPACE,AGIS]&reason=comment&expiration=2013-07-07T12:00:00
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.

  • Example:
  • Set Manual status: excluding 2 ddmendpoints (DDM1,DDM2) for permission flags u,w and s:
  •   -- https://atlas-agis-api.cern.ch/request/ddmendpointstatus/update/set_location_status/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2&activity=uws&value=OFF&reason=temporary_excluded
3. To retrieve the history of DDMEndpoint status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&ddmendpoint[]=DDM1&ddmendpoint[]=DDM2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&provider=[DISKSPACE|AGIS|SRMPing|manual]&activity=[d,f,r,s,u,w]
Examples,
The equivalents of DQ2 blacklisting functions:
  • DQ2.getStatusHistory(locations=['DDM1', 'DDM2'], activity=['u','w'])
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&preset=ddmcompat&show=manual&activity=rw&ddmendpoint[]=DDM1&ddmendpoint=DDM2 | JSON example
  • DQ2.getConditionHistory(locations=['DDM1', 'DDM2'], activity=['u','w'])
  •   --- http://atlas-agis-api.cern.ch/request/ddmendpointstatus/query/history/?json&preset=ddmcompat&show=auto&activity=rw&ddmendpoint[]=DDM1&ddmendpoint=DDM2 | JSON example