ATLAS Grid Information System

PANDA blacklisting data
Prioritized list of probes:manual (A) switcher (A) hammercloud (A) hctest1 (D)
State values:OFFLINE TEST BROKEROFF PAUSED ONLINE
Experiment SitePandaQueue
Status
default
AGLT2 AGLT2_HOSPITAL manual
AGLT2_LMEM_SL7-condor hammercloud
AGLT2_TEST manual
ANALY_AGLT2_VP manual
ANLASC ALCF_Theta manual
ALCF_Theta_ES_TEST manual
test_LCRC manual
test_LCRC_ES manual
ARNES ANALY_ARNES_DIRECT hammercloud
BNL-ATLAS ANALY_BNL_INTEL manual
ANALY_BNL_Test_2_CE_1 manual +
BNL_PROD_CONTR_TEST manual
BNL_PROD_INTEL manual
BNL_PROD_UCORE manual
BNL_Test_2_CE_1 hammercloud
ES_ORNL_Titan manual
HARVESTER_APF_BNL_TEST hammercloud
BNL-ATLAS-OPP ANALY_BNL_GPU_ARC manual +
ANALY_BNL_KNL manual
ATLAS_OPP_OSG-CIT_CMS_T2 manual
ATLAS_OPP_OSG-CLEMSON_PALMETTO manual
ATLAS_OPP_OSG-FNAL_GPCE01 manual
ATLAS_OPP_OSG-FNAL_GPCE02 manual
ATLAS_OPP_OSG-GLOW manual
ATLAS_OPP_OSG-HYAK manual
ATLAS_OPP_OSG-MIT_CMS manual
ATLAS_OPP_OSG-NYSGRID_CORNELL_NYS1 manual
ATLAS_OPP_OSG-SU_OG manual
ATLAS_OPP_OSG-USCMS-FNAL-WC1-CE3 manual
ATLAS_OPP_OSG_ES-PURDUE manual
ATLAS_OPP_OSG_ES-UCONN manual
ATLAS_OPP_OSG_MCORE-UCSDT2 manual
BNL_KNL_ES manual
BNL_KNL_MCORE manual +
BOINC ANALY_BOINC manual
BOINC-TEST manual
BOINC_MCORE manual
CA-TORONTO-T3 CA-TORONTO-T3 switcher +
CA-VICTORIA-WESTGRID-T2 ANALY_VICTORIA_TEST manual +
CA-VICTORIA-K8S-TEST-T2 manual +
CERN-EXTENSION ANALY_CERN-EXTENSION_KUBECON manual +
ANALY_CERN-EXTENSION_KUBECON2 manual +
ANALY_CERN_EXTENSION_TEST manual +
CERN-EXTENSION_DYNAFED manual +
CERN-EXTENSION_HARVESTER manual
CERN-EXTENSION_K8S_HARVESTER manual +
CERN-EXTENSION_KUBERNETES manual +
CERN-EXTENSION_KUBERNETES_TEST manual +
CERN-P1 ANALY_CERN-P1_COVID manual +
CERN-P1 manual +
CERN-P1_UCORE manual
CERN-PROD ALL manual
ANALY_CERN-PTEST manual +
ANALY_CERN_T0_ART manual
CERN-AZURE hammercloud
CERN-BACKFILL hammercloud
CERN-HPC manual +
CERN-PROD manual
CERN-PROD_DEV manual +
CERN-PROD_UCORE manual
CERN-PROD_UCORE_1 manual
CERN-PROD_UCORE_2 manual
CERN-PROD_UCORE_3 manual +
CERN-T0 ANALY_CERN_T0_COVID manual +
CERN-PROD_T0 manual
DESY-HH DESY-HH_GPU manual
DESY-HH_TEST manual
FR-ALPAMED-CPPM IN2P3-CPPM-TEST hammercloud
FR-ALPAMED-LAPP IN2P3-LAPP-TEST manual +
FR-ALPAMED-LPC IN2P3-LPC-TEST hammercloud
FZK-LCG2 FZK-LCG2_MCORE manual +
GOOGLE EKS manual +
GKE manual +
GOOGLE100 manual +
GoeGrid GoeGrid_GU switcher +
IFIC-LCG2 IFIC_ARC_TEST manual +
IFIC_Lusitania2 manual
IFIC_MareNostrum4 manual
IL-TAU-HEP IL-TAU hammercloud
IN2P3-CPPM IN2P3-CPPM-SE manual +
IN2P3-LAPP IN2P3-LAPP-SE manual +
IN2P3-LPC IN2P3-LPC-SE manual +
IN2P3-LPSC IN2P3-LPSC-SE manual +
IN2P3-LPSC-TEST hammercloud
INFN-COSENZA INFN-COSENZA-RECAS hammercloud
INFN-MILANO-ATLASC INFN-MILANO-ATLASC_TEST manual
INFN-T1 ANALY_INFN-T1_GPU manual
ANALY_INFN-T1_TEST_SL7 manual +
INFN-T1-CONDOR-TEST hammercloud
INFN-T1_UCORE manual
JINR-LCG2 JINR_UCORE-TEST manual +
LBNL_DSD_ITB NERSC_Cori_Test manual
NERSC_Cori_p2_ES manual
NERSC_Cori_p2_ES_Test manual
NERSC_Cori_p2_mcore manual
LRZ-LMU ANALY_LRZ_GPU hammercloud
ANALY_LRZ_VP manual
LRZ-LMU_C2PAP_ES_MCORE hammercloud
LRZ-LMU_CLOUD hammercloud
LRZ-LMU_MUC manual +
LRZ-LMU_MUC_MCORE manual
LRZ-LMU_TEST manual
LRZ-LMU_XCACHE manual
LUCILLE Lucille_CE switcher
Lucille_MCORE switcher
MWT2 ANALY_MWT2_GPU manual +
ANALY_MWT2_VP manual
IRISHEP-SSL-UCHICAGO manual
MWT2_K8S_UCORE manual +
NDGF-T1 ARC-TEST manual
OLCF ANALY_ORNL_Summit manual
ANALY_ORNL_Summit_GPU manual
ANALY_ORNL_Summit_ML manual
OU_OSCER_ATLAS ANALY_OU_OSCER_GPU_TEST manual +
OU_OSCER_ATLAS_TEST hammercloud
OU_OSCER_ATLAS_UCORE manual
RAL-LCG2 RAL-LCG2_ES hammercloud
RAL-LCG2_TEST manual +
RAL-LCG2_UCORE manual
RRC-KI-T1 RRC-KI-T1_TEST manual
RRC-KI-T1_UCORE manual
SARA-MATRIX SARA-MATRIX_TEST manual
SWT2_CPB SWT2_PAUL_TEST manual
SiGNET SiGNET_MCORE manual
TACC TACC-FRONTERA-UCORE manual +
TECHNION-HEP TECHNION hammercloud
TW-FTT ANALY_TAIWAN_TEST manual
UAM-LCG2 UAM_Cibeles manual
UKI-LT2-Brunel UKI-LT2-Brunel_ipv6_TEST hammercloud
UKI-LT2-QMUL ANALY_QMUL_GPU_TEST manual +
UKI-NORTHGRID-LANCS-HEP UKI-NORTHGRID-LANCS-HEP_TEST manual
UKI-NORTHGRID-MAN-HEP ANALY_MANC_GPU_TEST manual +
ANALY_MANC_TEST_SL7 manual +
UKI-NORTHGRID-MAN-HEP_TEST manual
UKI-NORTHGRID-SHEF-HEP UKI-NORTHGRID-SHEF-HEP hammercloud
UKI-SCOTGRID-GLASGOW UKI-SCOTGRID-GLASGOW_CEPH_TEST manual +
UKI-SOUTHGRID-SUSX UKI-SOUTHGRID-SUSX_UCORE manual
UNI-FREIBURG UNI-FREIBURG_MCORE manual
UNI-FREIBURG_TEST manual
UNI-SIEGEN-HEP UNI-SIEGEN-HEP manual
UNIBE-LHEP UNIGE-BAOBAB hammercloud
UTA_SWT2 UTA_PAUL_TEST manual
UTA_PAVLO_TEST manual
UTA_PAVLO_TEST_ARC manual
WEIZMANN-LCG2 WEIZMANN hammercloud
WISC-ATLAS WISC_ATLAS_TOKEN manual
WT2 ANALY_SLAC_GPU manual
ANALY_SLAC_SHORT_1HR manual
SLAC_ES hammercloud
pic pic-htcondor_UCORE manual
pic_MareNostrum4 manual +
pic_MareNostrum4_Harvester manual
pic_MareNostrum4_Test manual
praguelcg2 ANALY_FZU_VP manual
praguelcg2_fzu_TEST hammercloud
ru-Moscow-MEPHI-LCG2 MEPHI-TEST hammercloud
MEPHI_XCACHE manual +
ru-PNPI_XCACHE PNPI_XCACHE-TEST hammercloud
ru-PNPI_XCACHE_NODES PNPI_XCACHE-NODE hammercloud
ru-pnpi_NoXCACHE PNPI-TEST2 hammercloud
wuppertalprod wuppertalprod_CONDOR_TEST manual +
Documentation: TWiki | GET API doc | POST API doc
PandaBlacklisting HOWTO (Instructions for Shifters)
  • 0. Make sure that you have required permissions granted in AGIS: PANDA-ADMIN or ATLAS-FULL. If not then do Request ADMIN privileges.
    PQ status can be changed either from shell console using CURL command or directly from Web browser by opening special page.

  • Change PQ status from console (using CURL):
    • 1. Initialize X509 proxy environment (at the moment only full legacy globus proxy is accepted)

    • $ voms-proxy-init -voms atlas --old # create proxy
    • $ export X509_USER_PROXY=/tmp/x509up_uXXXXXXX ## setup environment to point your proxy location

    • 2. Perform curl query to AGIS (check the desctiption of URI parameters below)

    • $ curl --capath /etc/grid-security/certificates --cacert $X509_USER_PROXY --cert $X509_USER_PROXY 'https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual'

  • Change PQ status from Web browser:
    • 1. Ensure that your certificate is imported into a browser
    • 2. Construct and open URI like following to query AGIS:

    • https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual

  • Accepted URI parameters:
    • pandaqueue: name of PandaQueue for which status will be changed
    • site (optional): ATLASSite name, to blacklist all PQs for given site
    • value: status value, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, PAUSED, AUTO]
    • reason: comment message (spaces should be replaced by '+')
    • probe: type of issuer. Should be 'manual' for CLI
    • expiration: Expiration time (in UTC), mandatory field for 'manual' probe. Supports relative datetime value to current time in UTC with format='-?%d[w,d,h,m,s]' ~ weeks/days/hours/minutes/seconds (for example, expiration=5h means time=utc.now()+5 hours) or full datetime value in format like YYYY-MM-DD HH:MM[:ss]

  • Return codes:
    • For successfull operation AGIS returns following message (code=200 or code=304):
      Code=200 means that database has been successfully updated while code=304 indicates that no changes need to be applied (not modified)

      Return code: 200. Status of PanDA queue "NAME_OF_PANDAQUE" was set to "VALUE_OF_STATUS" for probe="manual", activity="a", expiration=None

    • In case of errors (code=500):

      Return code: [error code]. [Error message]

PandaBlacklisting JSON API Reference
1. To retrieve (current) data in JSON format, filter by fstate (final state) or activity values:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED]
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[..]&activity=[a1,a2]
Accepted filters:
  • fstate: filter by final states, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, PAUSED] | JSON example
  • activity: filter by activities, possible values: not used yet.
  • probe: filter by probe/provider name, possible value=[manual, hammercloud, switcher] | JSON example
  • pandaqueue: filter by PandaQueue name
2. Curl like API (GET requests and POST for bulk updates) to update PandaQueue blacklisting data:
  • GET is accepted HTTP method to submit data
  • To update state of PandaQueue
    1. accepted keys: probe, pandaqueue, activity, value, reason, expiration
    2. expiration key is optional and can be omitted
    3. GET URI format:
    4. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&pandaqueue[]=PQ1&pandaqueue[]=PQ2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED|AUTO]&reason=put+comment+here&probe=manual&expiration=2018-01-01T12:10:00
  •  
  • To blacklist whole ATLAS site (set manual state for all PandaQueues attached to the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED|AUTO]&probe=[hammercloud,switcher,manual]&reason=bulk+blacklist+comment&expiration=2015-07-07T12:00:00
  •  
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.
3. To retrieve the history of PandaQueue status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&pandaqueue[]=PQ&pandaqueue[]=PQ2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&probe=[hammercloud|switched|manual]&activity=[a] | JSON example