ATLAS Grid Information System

PANDA blacklisting data
Prioritized list of probes:manual (A) switcher (A) hammercloud (A) hctest1 (D)
State values:OFFLINE TEST BROKEROFF PAUSED ONLINE
Experiment SitePandaQueue
Status
default
AGLT2 ANALY_AGLT2_TEST_SL6-condor hammercloud
ANLASC ALCF_Cooley manual
ALCF_Theta manual
ALCF_Theta_ES manual
ARNES ANALY_ARNES hammercloud
ANALY_ARNES_DIRECT hammercloud
Australia-ATLAS ANALY_AUSTRALIA_TEST manual
Australia-ATLAS_MCORE manual
Australia-ATLAS_MCORE_TEST manual
Australia-ATLAS_TEST manual
BEIJING-LCG2 BEIJING-CS-TH-1A_MCORE manual
BEIJING-ERAII_MCORE manual
BEIJING-TIANJIN-TH-1A_MCORE manual
BNL-ATLAS ANALY_BNL_Test_2_CE_1-condor hammercloud
BNL_PROD_CONTR_TEST-condor hammercloud
BNL_Test_2_CE_1-condor hammercloud
ES_ORNL_Titan manual
HARVESTER_APF_BNL_TEST hammercloud
BNL-ATLAS-OPP ANALY_BNL_CLOUD manual
ANALY_BNL_LOCAL manual
ANALY_LONG_BNL_LOCAL manual
ATLAS_OPP_OSG-CIT_CMS_T2 hammercloud
ATLAS_OPP_OSG-GLOW manual
ATLAS_OPP_OSG-SU_OG manual
ATLAS_OPP_OSG_ES-UCONN manual
BNL_CLOUD manual
BNL_KNL_MCORE manual +
BNL_LOCAL-condor manual
BOINC ANALY_BOINC manual
BOINC-TEST manual
BOINC_MCORE manual
CA-MCGILL-CLUMEQ-T2 ANALY_MCGILL switcher
ANALY_MCGILL_TEST manual
CA-MCGILL-CLUMEQ-T2 switcher
CA-MCGILL-CLUMEQ-T2_MCORE switcher
CA-SCINET-T2 ANALY_SCINET_TEST manual
CA-SFU-T2 ANALY_SFU_CONTAINER manual +
ANALY_SFU_TEST manual +
CA-SFU-T2_CONTAINER manual +
CA-SFU-T2_CONTAINER_MCORE manual +
CA-VICTORIA-WESTGRID-T2 ANALY_VICTORIA_TEST manual +
CERN-EXTENSION CERN-EXTENSION_CERN manual +
CERN-EXTENSION_DE manual +
CERN-EXTENSION_HARVESTER manual +
CERN-EXTENSION_TEST hammercloud
CERN-EXTENSION_UK manual +
CERN-PROD ANALY_CERN_T0_ART manual +
CERN-PROD_PUB_SCORE_SHORT manual
CSCS-LCG2 CSCS-LCG2-HPC_MCORE_TEST manual +
CYFRONET-LCG2 CYFRONET-LCG2-all-prod-CEs hammercloud
CYFRONET-LCG2-all-prod-CEs_MCORE hammercloud
FZK-LCG2 FZK-LCG2-all-prod-CEs manual
HELIX_NEBULA HELIX_NEBULA_ATOS manual
HELIX_NEBULA_EGI manual
HEPHY-UIBK HEPHY-UIBK-grid-atlas-pbs hammercloud
IAAS ANALY_IAAS manual
ANALY_IAAS_TEST manual
IAAS-cloudscheduler manual +
IAAS_MCORE manual +
IAAS_MCORE_HIMEM manual
IL-TAU-HEP ANALY_IL-TAU-HEP-CREAM hammercloud
IN2P3-CC IN2P3-CC_HPC_DEBUG manual +
IN2P3-CC_HPC_IDRIS_MCORE manual
IN2P3-CC_HPC_IDRIS_MCORE1 manual +
IN2P3-CC-T3 IN2P3-CC-T3_MCORE manual
IN2P3-CC-T3_VM01 manual
IN2P3-CC-T3_VM01_SL7 manual
IN2P3-CC-T3_VM02_SL7 manual
IN2P3-LAPP ANALY_LAPP_TEST hammercloud
IN2P3-LAPP-TEST hammercloud
INFN-BOLOGNA-T3 ANALY_INFN-BOLOGNA-T3 switcher
INFN-BOLOGNA-T3-cebo-t3-01-cream-lsf switcher
INFN-FRASCATI ANALY_INFN-FRASCATI_PODtest hammercloud
INFN-NAPOLI-ATLAS INFN-NAPOLI-RECAS-TEST hammercloud
INFN-T1 ANALY_INFN-T1 switcher
ANALY_INFN-T1-VWN_TEST manual
INFN-T1-EVGENSIM switcher
INFN-T1-all-prod-CEs switcher
INFN-T1_HIMEM switcher
INFN-T1_MCORE switcher
INFN-T1_TEST switcher
LBNL_DSD_ITB NERSC_Cori_p1_mcore manual +
NERSC_Cori_p2_ES manual
NERSC_Cori_p2_mcore manual
NERSC_Edison_mcore manual
LUCILLE Lucille_CE switcher +
Lucille_MCORE switcher +
MWT2 ANALY_CONNECT manual
ANALY_CONNECT_SHORT manual
ANALY_CONNECT_TEST manual
CONNECT manual
CONNECT_BLUEWATERS manual
CONNECT_BLUEWATERS_MCORE manual
CONNECT_ES manual
CONNECT_ES_BLUEWATERS manual
CONNECT_ES_BLUEWATERS_MCORE manual
CONNECT_ES_ICC manual +
CONNECT_ES_JETSTREAM_MCORE manual +
CONNECT_ES_MCORE manual
CONNECT_ES_ODYSSEY manual +
CONNECT_ES_ODYSSEY_MCORE manual +
CONNECT_ES_STAMPEDE manual +
CONNECT_ES_STAMPEDE_MCORE manual +
CONNECT_ES_UIUC manual
CONNECT_ES_UIUC_MCORE manual
CONNECT_MCORE manual
CONNECT_MIDWAY manual
CONNECT_MIDWAY_MCORE manual
CONNECT_ODYSSEY manual
CONNECT_ODYSSEY_MCORE hammercloud
CONNECT_STAMPEDE manual +
CONNECT_STAMPEDE_MCORE manual +
CONNECT_UIUC manual
CONNECT_UIUC_MCORE manual
NDGF-T1 ARC-TEST manual
UIO_ARC_TEST manual +
UIO_CLOUD manual +
OLCF ORNL_Titan_MCORE manual
Titan_Harvester_MCORE manual
Titan_Harvester_test_MCORE manual
OU_OCHEP_SWT2 OU_OCHEP_SWT2-condor manual +
OU_OSCER_ATLAS OU_OSCER_ATLAS manual
OU_OSCER_ATLAS_OPP manual
OU_OSCER_ATLAS_TEST manual +
PSNC PSNC-creamce-atlas-pbs_MCORE manual
RAL-Azure RAL-AZURE_VAC manual +
RAL-LCG2 ANALY_RAL_MCORE manual
RAL-LCG2_SL6 manual
RAL-LCG2_VAC manual +
RAL-LCG2-ECHO RAL-LCG2-ECHO manual
RRC-KI ANALY_RRC-KI switcher +
RRC-KI_MCORE switcher
RRC-KI_PROD switcher
RRC-KI_TEST_DAN manual
RRC-KI-T1 RRC-KI-T1_TEST manual
SE-SNIC-T2 LUNARC_TEST hammercloud
TECHNION-HEP ANALY_TECHNION-HEP-CREAM hammercloud
TRIUMF-LCG2 ANALY_TRIUMF_DOCKER manual +
ANALY_TRIUMF_HIMEM manual
ANALY_TRIUMF_PPS manual
ANALY_TRIUMF_SL7 manual +
ANALY_TRIUMF_TEST manual
TRIUMF_DOCKER_UCORE manual +
TRIUMF_PPS manual
TRIUMF_SL7 manual +
TRIUMF_SL7_MCORE manual
TRIUMF_TEST manual
TW-FTT ANALY_TAIWAN_XROOTD_SL6 manual
Taiwan-LCG2 Taiwan-LCG2-HPC_HIMEM-all-prod-CEs manual
UAM-LCG2 ANALY_UAM switcher
UAM-LCG2_CREAM_HTCONDOR switcher
UAM-LCG2_CREAM_HTCONDOR_MCORE switcher
UKI-LT2-IC-HEP UKI-LT2-IC-HEP_TEST_SL7 hammercloud
UKI-LT2-QMUL UKI-LT2-QMUL_TEST manual
UKI-NORTHGRID-LANCS-HEP UKI-NORTHGRID-LANCS-HEP_ES manual
UKI-NORTHGRID-LANCS-HEP_TEST hammercloud
UKI-NORTHGRID-MAN-HEP UKI-NORTHGRID-MAN-HEP_SL6 manual +
UKI-NORTHGRID-MAN-HEP_TEST_SL7 hammercloud
UKI-SCOTGRID-ECDF ANALY_ECDF_SL6 manual
UKI-ARCHER_MCORE manual
UKI-SCOTGRID-GLASGOW UKI-SCOTGRID-GLASGOW_TEST manual
UKI-SOUTHGRID-OX-HEP ANALY_OX_TEST hammercloud
UKI-SOUTHGRID-OX-HEP_TEST manual
UKI-SOUTHGRID-SUSX ANALY_SUSX_SL6 manual
UNIBE-LHEP UNIBE-LHEP_CLOUD hammercloud
UNIBE-LHEP_CLOUD_MCORE hammercloud
UNIGE-DPNC UNIGE-DPNC manual
UNIGE-DPNC_MCORE manual
UTA_SWT2 UTA_PAUL_TEST manual
WEIZMANN-LCG2 ANALY_WEIZMANN-CREAM hammercloud
WT2 ANALY_SLAC_LMEM-lsf manual
ANALY_SLAC_SHORT_1HR-lsf manual
ZA-UJ ANALY_ZA-UJ hammercloud
ZA-UJ-glite-ce-pbs hammercloud
pic pic-ipv6-test manual
praguelcg2 Anselm_MCORE manual
Documentation: TWiki | GET API doc | POST API doc
PandaBlacklisting HOWTO (Instructions for Shifters)
  • 0. Make sure that you have required permissions granted in AGIS: PANDA-ADMIN or ATLAS-FULL. If not then do Request ADMIN privileges.
    PQ status can be changed either from shell console using CURL command or directly from Web browser by opening special page.

  • Change PQ status from console (using CURL):
    • 1. Initialize X509 proxy environment (at the moment only full legacy globus proxy is accepted)

    • $ voms-proxy-init -voms atlas --old # create proxy
    • $ export X509_USER_PROXY=/tmp/x509up_uXXXXXXX ## setup environment to point your proxy location

    • 2. Perform curl query to AGIS (check the desctiption of URI parameters below)

    • $ curl --capath /etc/grid-security/certificates --cacert $X509_USER_PROXY --cert $X509_USER_PROXY 'https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual'

  • Change PQ status from Web browser:
    • 1. Ensure that your certificate is imported into a browser
    • 2. Construct and open URI like following to query AGIS:

    • https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual

  • Accepted URI parameters:
    • pandaqueue: name of PandaQueue for which status will be changed
    • site (optional): ATLASSite name, to blacklist all PQs for given site
    • value: status value, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, PAUSED, AUTO]
    • reason: comment message (spaces should be replaced by '+')
    • probe: type of issuer. Should be 'manual' for CLI
    • expiration: Expiration time (in UTC), mandatory field for 'manual' probe. Supports relative datetime value to current time in UTC with format='-?%d[w,d,h,m,s]' ~ weeks/days/hours/minutes/seconds (for example, expiration=5h means time=utc.now()+5 hours) or full datetime value in format like YYYY-MM-DD HH:MM[:ss]

  • Return codes:
    • For successfull operation AGIS returns following message (code=200 or code=304):
      Code=200 means that database has been successfully updated while code=304 indicates that no changes need to be applied (not modified)

      Return code: 200. Status of PanDA queue "NAME_OF_PANDAQUE" was set to "VALUE_OF_STATUS" for probe="manual", activity="a", expiration=None

    • In case of errors (code=500):

      Return code: [error code]. [Error message]

PandaBlacklisting JSON API Reference
1. To retrieve (current) data in JSON format, filter by fstate (final state) or activity values:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED]
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[..]&activity=[a1,a2]
Accepted filters:
  • fstate: filter by final states, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, PAUSED] | JSON example
  • activity: filter by activities, possible values: not used yet.
  • probe: filter by probe/provider name, possible value=[manual, hammercloud, switcher] | JSON example
  • pandaqueue: filter by PandaQueue name
2. Curl like API (GET requests and POST for bulk updates) to update PandaQueue blacklisting data:
  • GET is accepted HTTP method to submit data
  • To update state of PandaQueue
    1. accepted keys: probe, pandaqueue, activity, value, reason, expiration
    2. expiration key is optional and can be omitted
    3. GET URI format:
    4. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&pandaqueue[]=PQ1&pandaqueue[]=PQ2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED|AUTO]&reason=put+comment+here&probe=manual&expiration=2018-01-01T12:10:00
  •  
  • To blacklist whole ATLAS site (set manual state for all PandaQueues attached to the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED|AUTO]&probe=[hammercloud,switcher,manual]&reason=bulk+blacklist+comment&expiration=2015-07-07T12:00:00
  •  
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.
3. To retrieve the history of PandaQueue status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&pandaqueue[]=PQ&pandaqueue[]=PQ2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&probe=[hammercloud|switched|manual]&activity=[a] | JSON example