ATLAS Grid Information System

PANDA blacklisting data
Prioritized list of probes:manual (A) switcher (A) hammercloud (A) hctest1 (D)
State values:OFFLINE TEST BROKEROFF PAUSED ONLINE
Experiment SitePandaQueue
Status
default
AGLT2 AGLT2_HOSPITAL manual
AGLT2_LMEM_SL7-condor hammercloud
AGLT2_TEST manual
ANLASC ALCF_Theta manual
ALCF_Theta_ES manual
ALCF_Theta_ES_TEST manual
Australia-ATLAS Australia-ATLAS_TEST manual
BNL-ATLAS ANALY_BNL_INTEL manual
ANALY_BNL_Test_2_CE_1 manual +
BNL_PROD_CONTR_TEST manual
BNL_Test_2_CE_1 hammercloud
ES_ORNL_Titan manual
HARVESTER_APF_BNL_TEST hammercloud
BNL-ATLAS-OPP ANALY_BNL_GPU_ARC manual +
ANALY_BNL_KNL manual
ATLAS_OPP_OSG-CIT_CMS_T2 manual
ATLAS_OPP_OSG-CLEMSON_PALMETTO manual
ATLAS_OPP_OSG-FNAL_GPCE01 manual
ATLAS_OPP_OSG-FNAL_GPCE02 manual
ATLAS_OPP_OSG-GLOW manual
ATLAS_OPP_OSG-HYAK manual
ATLAS_OPP_OSG-MIT_CMS manual
ATLAS_OPP_OSG-NYSGRID_CORNELL_NYS1 manual
ATLAS_OPP_OSG-SU_OG manual
ATLAS_OPP_OSG-USCMS-FNAL-WC1-CE3 manual
ATLAS_OPP_OSG_ES-PURDUE manual
ATLAS_OPP_OSG_ES-UCONN manual
ATLAS_OPP_OSG_MCORE-UCSDT2 manual
BNL_KNL_ES manual
BNL_KNL_MCORE manual +
BNLLAKE ANALY_BNL_LAKE hammercloud
BOINC ANALY_BOINC manual
BOINC-TEST manual
BOINC_BACKFILL manual
BOINC_MCORE manual
CA-TRIUMF-DYNAFED ANALY_TRIUMF_DYNAFED manual +
CA-VICTORIA-WESTGRID-T2 ANALY_VICTORIA_TEST manual +
CA-VICTORIA-K8S-TEST-T2 manual +
CA-WATERLOO-T2 ANALY_WATERLOO switcher
CERN-EXTENSION ANALY_CERN_EXTENSION_TEST manual +
ANALY_GOOGLE_HARVESTER manual
CERN-EXTENSION_DYNAFED manual +
CERN-EXTENSION_GOOGLE_HARVESTER manual +
CERN-EXTENSION_HARVESTER manual
CERN-EXTENSION_K8S_HARVESTER manual +
CERN-EXTENSION_KUBECON manual +
CERN-EXTENSION_KUBERNETES manual +
CERN-EXTENSION_KUBERNETES_TEST hammercloud
CERN-P1 CERN-P1 manual +
CERN-P1_SCORE manual +
CERN-P1_SCORE_HIMEM manual +
CERN-PROD ANALY_CERN manual
ANALY_CERN_GOOGLE_HARVESTER hammercloud
ANALY_CERN_HI manual
ANALY_CERN_T0_ART manual
CERN-AZURE hammercloud
CERN-BACKFILL hammercloud
CERN-HPC manual +
CERN-HPC_ES manual
CERN-PROD manual
CERN-PROD_DEV manual +
CERN-PROD_ES manual
CERN-PROD_HI manual
CERN-PROD_UCORE manual
CERN-PROD_UCORE_1 manual
CERN-PROD_UCORE_2 manual
CERN-PROD_UNI manual +
CERN-T0 T0 manual
CSCS-LCG2 ANALY_CSCS manual
DESY-HH ANALY_DESY-HH_TEST manual
DESY-HH_TEST manual
FR-ALPES ANALY_CPPM_TEST hammercloud
ANALY_LAPP_TEST hammercloud
ANALY_LPC_TEST hammercloud
ANALY_LPSC_TEST hammercloud
IN2P3-CPPM-TEST hammercloud
IN2P3-LAPP-TEST hammercloud
IN2P3-LPSC-TEST hammercloud
LPC_UCORE_TEST hammercloud
FZK-LCG2 FZK-LCG2_MCORE manual +
GoeGrid GoeGrid_GU switcher +
IAAS ANALY_IAAS manual
CA-IAAS-T3 manual
IFIC-LCG2 IFIC_ARC_TEST manual +
IFIC_Lusitania2 manual
IFIC_MareNostrum4 manual
INFN-COSENZA INFN-COSENZA-RECAS manual
INFN-LECCE ANALY_INFN-LECCE hammercloud
INFN-MILANO-ATLASC ANALY_INFN-MILANO-ATLASC hammercloud
INFN-ROMA1 INFN-ROMA1 manual
INFN-ROMA3 INFN-ROMA3 hammercloud
INFN-T1 ANALY_INFN-T1_GPU manual
ANALY_INFN-T1_TEST_SL7 manual +
INFN-T1-CINECA_HPC hammercloud
INFN-T1-CONDOR-TEST hammercloud
JINR-LCG2 JINR_UCORE manual +
JINR_UCORE-TEST manual +
LBNL_DSD_ITB NERSC_Cori_p1_mcore hammercloud
NERSC_Cori_p2_ES manual
NERSC_Cori_p2_ES_Test manual
NERSC_Cori_p2_mcore manual
LRZ-LMU ANALY_LRZ_GPU hammercloud
ANALY_LRZ_XCACHE hammercloud
LRZ-LMU_C2PAP_ES_MCORE hammercloud
LRZ-LMU_TEST manual
LRZ-LMU_UCORE hammercloud
LRZ-LMU_XCACHE manual
LUCILLE Lucille_CE switcher
Lucille_MCORE switcher
MWT2 ANALY_MWT2_GPU manual +
ANALY_MWT2_VP manual +
IRISHEP-SSL-UCHICAGO manual
MWT2_HOSPITAL manual +
MWT2_K8S_UCORE manual +
NDGF-T1 ANALY_DCSC hammercloud
ARC-TEST manual
OLCF ANALY_ORNL_Summit manual
ANALY_Summit_Harvester_MCORE manual
OU_OCHEP_SWT2 OUHEP_OSG manual
OU_OCHEP_SWT2 manual
OU_OSCER_ATLAS ANALY_OU_OSCER_GPU_TEST manual +
OU_OSCER_ATLAS_TEST hammercloud
RAL-LCG2 RAL-LCG2_ES manual +
RAL-LCG2_TEST manual +
RO-07-NIPNE ANALY_ROMANIA07_ARC_CL7 hammercloud
RRC-KI ANALY_RRC-KI manual
RRC-KI_MCORE manual
RRC-KI_PROD manual
RRC-KI-T1 RRC-KI-T1 manual
RRC-KI-T1_TEST manual +
SARA-MATRIX ANALY_SARA manual
SE-SNIC-T2 LUNARC_TEST hammercloud
SWT2_CPB SWT2_CPB_UCORE hammercloud
SWT2_PAUL_TEST manual
TECHNION-HEP ANALY_TECHNION-HEP-CREAM hammercloud
TECHNION-HEP manual +
TECHNION-HEP_MCORE manual +
TOKYO-LCG2 TOKYO_HPC hammercloud
TR-10-ULAKBIM ANALY_TR-10-ULAKBIM manual
TR-10-ULAKBIM hammercloud
TR-10-ULAKBIM_MCORE hammercloud
TRIUMF-LCG2 ANALY_TRIUMF_TEST manual
TW-FTT ANALY_TAIWAN_TEST manual
UAM-LCG2 UAM-LCG2_ARC_HTCONDOR manual +
UAM_Cibeles manual
UKI-LT2-Brunel UKI-LT2-Brunel_ipv6_TEST hammercloud
UKI-LT2-IC-HEP UKI-LT2-IC-HEP_SL7_UCORE hammercloud
UKI-LT2-QMUL ANALY_QMUL_GPU_TEST hammercloud
UKI-LT2-RHUL ANALY_RHUL_SL7 hammercloud
UKI-NORTHGRID-LANCS-HEP UKI-NORTHGRID-LANCS-HEP_TEST manual
UKI-NORTHGRID-MAN-HEP ANALY_MANC_GPU_TEST hammercloud
ANALY_MANC_TEST_SL7 manual +
UKI-NORTHGRID-MAN-HEP_TEST_SL7 manual +
UKI-SCOTGRID-GLASGOW UKI-SCOTGRID-GLASGOW_CEPH_TEST hammercloud
UKI-SOUTHGRID-CAM-HEP UKI-SOUTHGRID-CAM-HEP_MCOREVAC hammercloud
UKI-SOUTHGRID-SUSX UKI-SOUTHGRID-SUSX_UCORE manual
UNI-SIEGEN-HEP UNI-SIEGEN-HEP manual
UNIBE-LHEP ANALY_UNIBE-LHEP hammercloud
ANALY_UNIBE-LHEP-UBELIX hammercloud
UNIBE-LHEP_MCORE hammercloud
UTA_SWT2 UTA_PAUL_TEST manual
UTA_PAVLO_TEST manual
UTA_PAVLO_TEST_ARC manual
WT2 ANALY_SLAC_LMEM manual
ANALY_SLAC_SHORT_1HR manual
SLAC_ES hammercloud
ZA-CHPC ZA-CHPC_UCORE manual
ZA-WITS-CORE ZA-WITS-CORE hammercloud
ifae ifae_UNI manual
pic pic-htcondor_UCORE manual
pic_MareNostrum4 manual +
pic_MareNostrum4_Harvester manual
pic_MareNostrum4_Test manual
pic_UNI manual
praguelcg2 praguelcg2_fzu_TEST hammercloud
ru-PNPI_XCACHE PNPI_XCACHE-TEST hammercloud
ru-pnpi_NoXCACHE PNPI-TEST hammercloud
wuppertalprod wuppertalprod_CONDOR_TEST manual +
Documentation: TWiki | GET API doc | POST API doc
PandaBlacklisting HOWTO (Instructions for Shifters)
  • 0. Make sure that you have required permissions granted in AGIS: PANDA-ADMIN or ATLAS-FULL. If not then do Request ADMIN privileges.
    PQ status can be changed either from shell console using CURL command or directly from Web browser by opening special page.

  • Change PQ status from console (using CURL):
    • 1. Initialize X509 proxy environment (at the moment only full legacy globus proxy is accepted)

    • $ voms-proxy-init -voms atlas --old # create proxy
    • $ export X509_USER_PROXY=/tmp/x509up_uXXXXXXX ## setup environment to point your proxy location

    • 2. Perform curl query to AGIS (check the desctiption of URI parameters below)

    • $ curl --capath /etc/grid-security/certificates --cacert $X509_USER_PROXY --cert $X509_USER_PROXY 'https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual'

  • Change PQ status from Web browser:
    • 1. Ensure that your certificate is imported into a browser
    • 2. Construct and open URI like following to query AGIS:

    • https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?html&pandaqueue=ANALY_CERN_TEST&value=BROKEROFF&reason=Test+API+2&probe=manual

  • Accepted URI parameters:
    • pandaqueue: name of PandaQueue for which status will be changed
    • site (optional): ATLASSite name, to blacklist all PQs for given site
    • value: status value, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, PAUSED, AUTO]
    • reason: comment message (spaces should be replaced by '+')
    • probe: type of issuer. Should be 'manual' for CLI
    • expiration: Expiration time (in UTC), mandatory field for 'manual' probe. Supports relative datetime value to current time in UTC with format='-?%d[w,d,h,m,s]' ~ weeks/days/hours/minutes/seconds (for example, expiration=5h means time=utc.now()+5 hours) or full datetime value in format like YYYY-MM-DD HH:MM[:ss]

  • Return codes:
    • For successfull operation AGIS returns following message (code=200 or code=304):
      Code=200 means that database has been successfully updated while code=304 indicates that no changes need to be applied (not modified)

      Return code: 200. Status of PanDA queue "NAME_OF_PANDAQUE" was set to "VALUE_OF_STATUS" for probe="manual", activity="a", expiration=None

    • In case of errors (code=500):

      Return code: [error code]. [Error message]

PandaBlacklisting JSON API Reference
1. To retrieve (current) data in JSON format, filter by fstate (final state) or activity values:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED]
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/list/?json&fstate=[..]&activity=[a1,a2]
Accepted filters:
  • fstate: filter by final states, possible values=[ONLINE, OFFLINE, TEST, BROKEROFF, PAUSED] | JSON example
  • activity: filter by activities, possible values: not used yet.
  • probe: filter by probe/provider name, possible value=[manual, hammercloud, switcher] | JSON example
  • pandaqueue: filter by PandaQueue name
2. Curl like API (GET requests and POST for bulk updates) to update PandaQueue blacklisting data:
  • GET is accepted HTTP method to submit data
  • To update state of PandaQueue
    1. accepted keys: probe, pandaqueue, activity, value, reason, expiration
    2. expiration key is optional and can be omitted
    3. GET URI format:
    4. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&pandaqueue[]=PQ1&pandaqueue[]=PQ2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED|AUTO]&reason=put+comment+here&probe=manual&expiration=2018-01-01T12:10:00
  •  
  • To blacklist whole ATLAS site (set manual state for all PandaQueues attached to the site), for specific condition (by various providers):
    1. GET URI format:
    2. -- https://atlas-agis-api.cern.ch/request/pandaqueuestatus/update/set_probestatus/?json&site[]=ATLASSite1&site[]=ATLASSite2&activity=[a]&value=[ONLINE|OFFLINE|TEST|BROKEROFF|PAUSED|AUTO]&probe=[hammercloud,switcher,manual]&reason=bulk+blacklist+comment&expiration=2015-07-07T12:00:00
  •  
  • RETURN codes:
  • -- API response with "true" message indicates that operation has been successfully performed, "false" message reports that operation is successfuly executed but no changes found to be commited into the DB. In case of errors, error message will be returned.
3. To retrieve the history of PandaQueue status changes in JSON format:
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&pandaqueue[]=PQ&pandaqueue[]=PQ2 | JSON example
  • - http://atlas-agis-api.cern.ch/request/pandaqueuestatus/query/history/?json&probe=[hammercloud|switched|manual]&activity=[a] | JSON example