ATLAS Grid Information System

PANDA blacklisting data
Prioritized list of probes:manual (A) switcher (A) hammercloud (A) hctest1 (D)
State values:OFFLINE TEST BROKEROFF ONLINE
Experiment SitePandaQueue
Status
default
AGLT2 ANALY_AGLT2_TEST_SL6-condor hammercloud
ANLASC ALCF_Cooley manual
ALCF_Theta manual
Australia-ATLAS ANALY_AUSTRALIA_TEST manual
Australia-ATLAS_MCORE_TEST manual
Australia-ATLAS_TEST manual
BNL-ATLAS ANALY_BNL_Test_2_CE_1-condor hammercloud
ANALY_ORNL_Titan manual
BNL_Test_2_CE_1-condor hammercloud
ES_ORNL_Titan manual
HARVESTER_APF_BNL_TEST hammercloud
BNL-ATLAS-OPP ANALY_BNL_CLOUD manual
ANALY_BNL_LOCAL manual
ANALY_LONG_BNL_LOCAL manual
ATLAS_OPP_OSG-CIT_CMS_T2 hammercloud
BNL_CLOUD manual
BNL_LOCAL-condor manual
BOINC BOINC-TEST manual
BOINC_CHECKPOINT manual
BOINC_MCORE manual
CA-MCGILL-CLUMEQ-T2 ANALY_MCGILL_DOCKER manual
ANALY_MCGILL_TEST manual
CA-MCGILL-CLUMEQ-T2_DOCKER manual +
CA-SCINET-T2 ANALY_SCINET_TEST manual
CA-SFU-T2 ANALY_SFU_TEST manual +
CA-VICTORIA-WESTGRID-T2 ANALY_VICTORIA_TEST manual +
CERN-EXTENSION CERN-EXTENSION_HARVESTER manual +
CERN-EXTENSION_MCORE manual
CERN-EXTENSION_TEST hammercloud
CERN-PROD ANALY_CERN_T0_ART manual +
ANALY_CERN_T0_SHORT manual
CERN-PROD_CLOUD_MCORE manual
CERN-PROD_T0_SCORE_SHORT manual
CSCS-LCG2 ANALY_CSCS switcher
ANALY_CSCS-HPC switcher
CSCS-LCG2-HPC switcher
CSCS-LCG2-HPC_MCORE switcher
CSCS-LCG2-all-prod-CEs switcher
CSCS-LCG2-all-prod-CEs_MCORE switcher
DESY-HH ANALY_DESY-HH_TEST manual
DESY-ZN ANALY_DESY-ZN_XRD manual
HELIX_NEBULA HELIX_NEBULA_ATOS manual
HELIX_NEBULA_EGI manual
HEPHY-UIBK HEPHY-UIBK-grid-atlas-pbs switcher
HK-LCG2 HK-LCG2_ARC hammercloud
IAAS ANALY_IAAS manual
ANALY_IAAS_TEST manual
IAAS-cloudscheduler manual +
IAAS_MCORE manual +
IAAS_MCORE_HIMEM manual
IN2P3-CC IN2P3-CC_HPC_DEBUG manual +
IN2P3-CC_HPC_IDRIS_MCORE manual
IN2P3-CC_HPC_IDRIS_MCORE1 manual
IN2P3-CC-T3 IN2P3-CC-T3_MCORE manual
IN2P3-CC-T3_VM01 manual
IN2P3-CC-T3_VM01_SL7 manual
IN2P3-CC-T3_VM02 manual
IN2P3-CC-T3_VM02_SL7 manual
IN2P3-CPPM IN2P3-CPPM-all-ce-atlas-pbs hammercloud
IN2P3-CPPM_MCORE hammercloud
IN2P3-LAPP ANALY_LAPP_TEST hammercloud
IN2P3-LAPP-TEST hammercloud
INFN-FRASCATI ANALY_INFN-FRASCATI_PODtest hammercloud
INFN-MILANO-ATLASC ANALY_INFN-MILANO-ATLASC switcher
INFN-MILANO-ATLASC-all-prod-CEs switcher
INFN-MILANO-ATLASC_MCORE switcher
INFN-NAPOLI-ATLAS INFN-NAPOLI-RECAS-TEST hammercloud
INFN-NAPOLI-SCOPE-atlasce02-sl6 switcher
INFN-ROMA1 ANALY_INFN-ROMA1 switcher
INFN-ROMA1 switcher
INFN-ROMA1_MCORE switcher
INFN-T1 ANALY_INFN-T1-VWN2_TEST manual
ANALY_INFN-T1-VWN_TEST manual
LBNL_DSD_ITB NERSC_Cori_p1_ES manual
NERSC_Cori_p1_mcore manual
NERSC_Cori_p2_ES manual
NERSC_Cori_p2_mcore manual
NERSC_Edison_ES manual
NERSC_Edison_mcore manual
LRZ-LMU LRZ-LMU_C2PAP_MCORE manual
LRZ-LMU_MUC_MCORE1 manual
MWT2 CONNECT_ES_ODYSSEY manual +
CONNECT_ES_ODYSSEY_MCORE manual +
CONNECT_MIDWAY manual
CONNECT_MIDWAY_MCORE manual
CONNECT_ODYSSEY_MCORE hammercloud
NCG-INGRID-PT ANALY_NCG-INGRID-PT_SL6 switcher
NCG-INGRID-PT-ce04-atlasgrid-sge switcher
NCG-INGRID-PT_MCORE switcher
NDGF-T1 ARC-ES manual
ARC-TEST manual +
UIO_CLOUD manual +
OLCF ORNL_Titan_MCORE manual
Titan_Harvester_MCORE manual
OU_OCHEP_SWT2 OUHEP_OSG switcher
OU_OSCER_ATLAS OU_OSCER_ATLAS_TEST hammercloud
PSNC PSNC-creamce-atlas-pbs_MCORE manual
RAL-Azure RAL-AZURE_VAC manual +
RAL-LCG2 ANALY_RAL_MCORE manual
RAL-LCG2_MCORE manual +
RO-02-NIPNE ANALY_ROMANIA02 switcher
RO-02-NIPNE-tbat03-atlas-lcgpbs switcher
RO-02-NIPNE-tbat03-atlas-lcgpbs_MCORE switcher
RRC-KI RRC-KI_TEST_DAN manual
RRC-KI-T1 RRC-KI-T1_TEST manual
SARA-MATRIX ANALY_SARA_bignode manual
ANALY_SARA_hpc_cloud manual
ANALY_SARA_hpc_cloud_rfio manual
ANALY_SARA_hpc_cloud_xrootd manual
SARA-MATRIX_LONG-all-prod-CEs hammercloud
SE-SNIC-T2 LUNARC_TEST hammercloud
TRIUMF-LCG2 ANALY_TRIUMF_DOCKER manual +
ANALY_TRIUMF_HIMEM manual
ANALY_TRIUMF_PPS manual
ANALY_TRIUMF_SL7 manual +
ANALY_TRIUMF_TEST manual
TRIUMF_DOCKER manual +
TRIUMF_DOCKER_MCORE manual +
TRIUMF_PPS manual
TRIUMF_SL7 manual +
TRIUMF_SL7_MCORE manual
TRIUMF_TEST manual
TW-FTT ANALY_TAIWAN_TEST manual
ANALY_TAIWAN_XROOTD_SL6 manual
Taiwan-LCG2 Taiwan-LCG2-HPC_HIMEM-all-prod-CEs manual
Taiwan-LCG2-HPC_VL-all-prod-CEs hammercloud
UKI-GRIDPP-CLOUD-IC GRIDPP_CLOUD manual
GRIDPP_MCORE manual
UKI-LT2-QMUL UKI-LT2-QMUL_TEST manual
UKI-NORTHGRID-LIV-HEP ANALY_LIV_SL6 hammercloud
UKI-NORTHGRID-LIV-HEP_MCORE hammercloud
UKI-NORTHGRID-LIV-HEP_SL6 hammercloud
UKI-NORTHGRID-LIV-HEP_SL7 hammercloud
UKI-NORTHGRID-LIV-HEP_VAC hammercloud
UKI-NORTHGRID-MAN-HEP UKI-NORTHGRID-MAN-HEP_SL6 manual +
UKI-NORTHGRID-MAN-HEP_TEST_SL7 hammercloud
UKI-SCOTGRID-ECDF ANALY_ECDF_SL7 manual
UKI-ARCHER_MCORE manual
UKI-SCOTGRID-ECDF_TEST hammercloud
UKI-SCOTGRID-GLASGOW UKI-SCOTGRID-GLASGOW_TEST manual
UKI-SOUTHGRID-OX-HEP ANALY_OX_TEST hammercloud
UKI-SOUTHGRID-OX-HEP_TEST manual
UKI-SOUTHGRID-SUSX ANALY_SUSX_SL6 manual
UNIBE-LHEP UNIBE-LHEP_CLOUD switcher +
UNIBE-LHEP_CLOUD_MCORE switcher +
UTA_SWT2 UTA_PAUL_TEST manual
WT2 ANALY_SLAC_LMEM-lsf manual
ANALY_SLAC_SHORT_1HR-lsf manual
ZA-UJ ANALY_ZA-UJ hammercloud
ZA-UJ-glite-ce-pbs hammercloud
pic pic-ipv6-test manual
praguelcg2 ANALY_praguelcg2_IT4I_MCORE_TEST manual +
Anselm_MCORE manual
Documentation: TWiki | GET API doc | POST API doc
PandaBlacklisting HOWTO (Instructions for Shifters)
  • 0. Make sure that you have required permissions granted in AGIS: PANDA-ADMIN or ATLAS-FULL. If not then do Request ADMIN privileges.
    PQ status can be changed either from shell console using CURL command or directly from Web browser by opening special page.

  • Change PQ status from console (using CURL):
    • 1. Initialize X509 proxy environment (at the moment only full legacy globus proxy is accepted)

    • $ voms-proxy-init -voms atlas --old # create proxy
    • $ export X509_USER_PROXY=/tmp/x509up_uXXXXXXX ## setup environment to point your proxy location

    • 2. Perform curl query to AGIS (check the desctiption of URI parameters below)

    • $ curl --capath /etc/grid-security/certificates --cacert $X509_USER_PROXY --cert $X509_USER_PROXY 'https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual'

  • Change PQ status from Web browser:
    • 1. Ensure that your certificate is imported into a browser
    • 2. Construct and open URI like following to query AGIS:

    • https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual

  • Accepted URI parameters:
    • pandaqueue: name of PandaQueue for which status will be changed
    • site (optional): ATLASSite name, to blacklist all PQs for given site
    • value: status value, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, AUTO]
    • reason: comment message (spaces should be replaced by '+')
    • probe: type of issuer. Should be 'manual' for CLI
    • expiration: Expiration time (in UTC), mandatory field for 'manual' probe. Supports relative datetime value to current time in UTC with format='-?%d[w,d,h,m,s]' ~ weeks/days/hours/minutes/seconds (for example, expiration=5h means time=utc.now()+5 hours) or full datetime value in format like YYYY-MM-DD HH:MM[:ss]

  • Return codes:
    • For successfull operation AGIS returns following message (code=200 or code=304):
      Code=200 means that database has been successfully updated while code=304 indicates that no changes need to be applied (not modified)

      Return code: 200. Status of PanDA queue "NAME_OF_PANDAQUE" was set to "VALUE_OF_STATUS" for probe="manual", activity="a", expiration=None

    • In case of errors (code=500):

      Return code: [error code]. [Error message]

PandaBlacklisting JSON API Reference
1. To retrieve (current) data in JSON format, filter by fstate (final state) or activity values:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[ONLINE|OFFLINE|TEST|BROKEROFF]
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[..]&activity=[a1,a2]
Accepted filters:
  • fstate: filter by final states, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF] | JSON example
  • activity: filter by activities, possible values: not used yet.
  • probe: filter by probe/provider name, possible value=[manual, hammercloud, switcher] | JSON example
  • pandaqueue: filter by PandaQueue name
2. Curl like API (GET requests and POST for bulk updates) to update PandaQueue blacklisting data:
  • GET is accepted HTTP method to submit data
  • To update state of PandaQueue
    1. accepted keys: probe, pandaqueue, activity, value, reason, expiration
    2. expiration key is optional and can be omitted
    3. GET URI format:
    4. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&pandaqueue[]=PQ1&pandaqueue[]=PQ2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|AUTO]&reason=put+comment+here&probe=manual&expiration=2018-01-01T12:10:00
  •  
  • To blacklist whole ATLAS site (set manual state for all PandaQueues attached to the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|AUTO]&probe=[hammercloud,switcher,manual]&reason=bulk+blacklist+comment&expiration=2015-07-07T12:00:00
  •  
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.
3. To retrieve the history of PandaQueue status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&pandaqueue[]=PQ&pandaqueue[]=PQ2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&probe=[hammercloud|switched|manual]&activity=[a] | JSON example