public class ZookeeperFederationStateStore extends Object implements FederationStateStore
FederationStateStore
.
The znode structure is as follows:
ROOT_DIR_PATH
|--- MEMBERSHIP
| |----- SC1
| |----- SC2
|--- APPLICATION
| |----- HIERARCHIES
| | |----- 1
| | | |----- (#ApplicationId barring last character)
| | | | | |----- APP Data
| | | ....
| | |
| | |----- 2
| | | |----- (#ApplicationId barring last 2 characters)
| | | | |----- (#Last 2 characters of ApplicationId)
| | | | | |----- APP Data
|--- POLICY
| |----- QUEUE1
| |----- QUEUE1
|--- RESERVATION
| |----- RESERVATION1
| |----- RESERVATION2
|--- ROUTER_RM_DT_SECRET_MANAGER_ROOT
| |----- ROUTER_RM_DELEGATION_TOKENS_ROOT
| | |----- RM_DELEGATION_TOKEN_1
| | |----- RM_DELEGATION_TOKEN_2
| | |----- RM_DELEGATION_TOKEN_3
| |----- ROUTER_RM_DT_MASTER_KEYS_ROOT
| | |----- DELEGATION_KEY_1
| |----- ROUTER_RM_DT_SEQUENTIAL_NUMBERModifier and Type | Field and Description |
---|---|
protected static Version |
CURRENT_VERSION_INFO |
protected static String |
ROOT_ZNODE_NAME_VERSION |
static String |
ROUTER_APP_ROOT_HIERARCHIES |
Constructor and Description |
---|
ZookeeperFederationStateStore() |
Modifier and Type | Method and Description |
---|---|
AddApplicationHomeSubClusterResponse |
addApplicationHomeSubCluster(AddApplicationHomeSubClusterRequest request)
Register the home
SubClusterId of the newly submitted
ApplicationId . |
AddReservationHomeSubClusterResponse |
addReservationHomeSubCluster(AddReservationHomeSubClusterRequest request)
Register the home
SubClusterId of the newly submitted
ReservationId . |
void |
close()
Perform any cleanup operations of the StateStore.
|
DeletePoliciesConfigurationsResponse |
deleteAllPoliciesConfigurations(DeletePoliciesConfigurationsRequest request)
Delete all queue-to-policy configurations.
|
DeleteApplicationHomeSubClusterResponse |
deleteApplicationHomeSubCluster(DeleteApplicationHomeSubClusterRequest request)
Delete the mapping of home
SubClusterId of a previously submitted
ApplicationId . |
DeleteSubClusterPoliciesConfigurationsResponse |
deletePoliciesConfigurations(DeleteSubClusterPoliciesConfigurationsRequest request)
Delete PoliciesConfigurations.
|
DeleteReservationHomeSubClusterResponse |
deleteReservationHomeSubCluster(DeleteReservationHomeSubClusterRequest request)
Delete the mapping of home
SubClusterId of a previously submitted
ReservationId . |
void |
deleteStateStore()
We will clear the data in stateStore through the deleteStateStore method.
|
SubClusterDeregisterResponse |
deregisterSubCluster(SubClusterDeregisterRequest request)
Deregister a subcluster identified by
SubClusterId to
change state in federation. |
GetApplicationHomeSubClusterResponse |
getApplicationHomeSubCluster(GetApplicationHomeSubClusterRequest request)
Get information about the application identified by the input
ApplicationId . |
GetApplicationsHomeSubClusterResponse |
getApplicationsHomeSubCluster(GetApplicationsHomeSubClusterRequest request)
Get the
ApplicationHomeSubCluster list representing the mapping of
all submitted applications to it's home sub-cluster. |
int |
getCurrentKeyId()
Get Current KeyId.
|
Version |
getCurrentVersion()
Get the
Version of the underlying federation state store client. |
int |
getDelegationTokenSeqNum()
Get DelegationToken SeqNum.
|
RouterMasterKeyResponse |
getMasterKeyByDelegationKey(RouterMasterKeyRequest request)
ZookeeperFederationStateStore Supports Remove MasterKey.
|
ZKFederationStateStoreOpDurations |
getOpDurations() |
GetSubClusterPoliciesConfigurationsResponse |
getPoliciesConfigurations(GetSubClusterPoliciesConfigurationsRequest request)
Get a map of all queue-to-policy configurations.
|
GetSubClusterPolicyConfigurationResponse |
getPolicyConfiguration(GetSubClusterPolicyConfigurationRequest request)
Get the policy configuration for a given queue.
|
GetReservationHomeSubClusterResponse |
getReservationHomeSubCluster(GetReservationHomeSubClusterRequest request)
Get information about the Reservation identified by the input
ReservationId . |
GetReservationsHomeSubClusterResponse |
getReservationsHomeSubCluster(GetReservationsHomeSubClusterRequest request)
Get the
ReservationHomeSubCluster list representing the mapping of
all submitted Reservations to it's home sub-cluster. |
GetSubClusterInfoResponse |
getSubCluster(GetSubClusterInfoRequest request)
Get the membership information of subcluster as identified by
SubClusterId . |
GetSubClustersInfoResponse |
getSubClusters(GetSubClustersInfoRequest request)
Get the membership information of all the subclusters that are
currently participating in federation.
|
RouterRMTokenResponse |
getTokenByRouterStoreToken(RouterRMTokenRequest request)
The Router Supports GetTokenByRouterStoreToken.
|
int |
incrementCurrentKeyId()
The Router Supports incrementCurrentKeyId.
|
int |
incrementDelegationTokenSeqNum()
Increase SequenceNum.
|
void |
init(org.apache.hadoop.conf.Configuration conf)
Initialize the FederationStore.
|
Version |
loadVersion()
Load the version information from the federation state store.
|
SubClusterRegisterResponse |
registerSubCluster(SubClusterRegisterRequest request)
Register a subcluster by publishing capabilities as represented by
SubClusterInfo to indicate participation in federation. |
RouterMasterKeyResponse |
removeStoredMasterKey(RouterMasterKeyRequest request)
ZookeeperFederationStateStore Supports Remove MasterKey.
|
RouterRMTokenResponse |
removeStoredToken(RouterRMTokenRequest request)
ZookeeperFederationStateStore Supports Remove RMDelegationTokenIdentifier.
|
void |
setDelegationTokenSeqNum(int seqNum)
Set DelegationToken SeqNum.
|
SetSubClusterPolicyConfigurationResponse |
setPolicyConfiguration(SetSubClusterPolicyConfigurationRequest request)
Set the policy configuration for a given queue.
|
protected void |
storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
byte[] data) |
RouterMasterKeyResponse |
storeNewMasterKey(RouterMasterKeyRequest request)
ZookeeperFederationStateStore Supports Store NewMasterKey.
|
RouterRMTokenResponse |
storeNewToken(RouterRMTokenRequest request)
ZookeeperFederationStateStore Supports Store RMDelegationTokenIdentifier.
|
void |
storeVersion()
Store the Version information in federation state store.
|
SubClusterHeartbeatResponse |
subClusterHeartbeat(SubClusterHeartbeatRequest request)
Periodic heartbeat from a
ResourceManager participating in
federation to indicate liveliness. |
UpdateApplicationHomeSubClusterResponse |
updateApplicationHomeSubCluster(UpdateApplicationHomeSubClusterRequest request)
Update the home
SubClusterId of a previously submitted
ApplicationId . |
protected void |
updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
byte[] data) |
UpdateReservationHomeSubClusterResponse |
updateReservationHomeSubCluster(UpdateReservationHomeSubClusterRequest request)
Update the home
SubClusterId of a previously submitted
ReservationId . |
RouterRMTokenResponse |
updateStoredToken(RouterRMTokenRequest request)
ZookeeperFederationStateStore Supports Update RMDelegationTokenIdentifier.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
checkVersion
protected static final String ROOT_ZNODE_NAME_VERSION
@VisibleForTesting public static final String ROUTER_APP_ROOT_HIERARCHIES
protected static final Version CURRENT_VERSION_INFO
public void init(org.apache.hadoop.conf.Configuration conf) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationStateStore
init
in interface FederationStateStore
conf
- the cluster configurationorg.apache.hadoop.yarn.exceptions.YarnException
- if initialization failspublic void close() throws Exception
FederationStateStore
close
in interface FederationStateStore
Exception
- if cleanup failspublic AddApplicationHomeSubClusterResponse addApplicationHomeSubCluster(AddApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
SubClusterId
of the newly submitted
ApplicationId
. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure. If a
mapping for the application already existed, the SubClusterId
in
this response will return the existing mapping which might be different
from that in the AddApplicationHomeSubClusterRequest
.addApplicationHomeSubCluster
in interface FederationApplicationHomeSubClusterStore
request
- the request to register a new application with its home sub-cluster.AddApplicationHomeSubClusterRequest
containing the home
sub-cluster of the application. Otherwise, an exception reporting
reason for a failure.org.apache.hadoop.yarn.exceptions.YarnException
- indicates exceptions from yarn servers.public UpdateApplicationHomeSubClusterResponse updateApplicationHomeSubCluster(UpdateApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
SubClusterId
of a previously submitted
ApplicationId
. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.updateApplicationHomeSubCluster
in interface FederationApplicationHomeSubClusterStore
request
- the request to update the home sub-cluster of an
application.org.apache.hadoop.yarn.exceptions.YarnException
- indicates exceptions from yarn servers.public GetApplicationHomeSubClusterResponse getApplicationHomeSubCluster(GetApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
ApplicationId
.getApplicationHomeSubCluster
in interface FederationApplicationHomeSubClusterStore
request
- contains the application queriedApplicationHomeSubCluster
containing the application's home
subclusterorg.apache.hadoop.yarn.exceptions.YarnException
- indicates exceptions from yarn servers.public GetApplicationsHomeSubClusterResponse getApplicationsHomeSubCluster(GetApplicationsHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
ApplicationHomeSubCluster
list representing the mapping of
all submitted applications to it's home sub-cluster.getApplicationsHomeSubCluster
in interface FederationApplicationHomeSubClusterStore
request
- empty representing all applicationsorg.apache.hadoop.yarn.exceptions.YarnException
- indicates exceptions from yarn servers.public DeleteApplicationHomeSubClusterResponse deleteApplicationHomeSubCluster(DeleteApplicationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
SubClusterId
of a previously submitted
ApplicationId
. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.deleteApplicationHomeSubCluster
in interface FederationApplicationHomeSubClusterStore
request
- the request to delete the home sub-cluster of an
application.org.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic SubClusterRegisterResponse registerSubCluster(SubClusterRegisterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStore
SubClusterInfo
to indicate participation in federation. This is
typically done during initialization or restart/failover of the
subcluster's ResourceManager
. Upon successful registration, an
identifier for the subcluster which is unique across the federated
cluster is returned. The identifier is static, i.e. preserved across
restarts and failover.registerSubCluster
in interface FederationMembershipStateStore
request
- the capabilities of the subcluster that
wants to participate in federation. The subcluster id is also
specified in case registration is triggered by restart/failoverorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic SubClusterDeregisterResponse deregisterSubCluster(SubClusterDeregisterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStore
SubClusterId
to
change state in federation. This can be done to mark the sub cluster lost,
deregistered, or decommissioned.deregisterSubCluster
in interface FederationMembershipStateStore
request
- - the request to deregister the
sub-cluster from federation.org.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic SubClusterHeartbeatResponse subClusterHeartbeat(SubClusterHeartbeatRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStore
ResourceManager
participating in
federation to indicate liveliness. The heartbeat publishes the current
capabilities as represented by SubClusterInfo
of the subcluster.
Currently response is empty if the operation was successful, if not an
exception reporting reason for a failure.subClusterHeartbeat
in interface FederationMembershipStateStore
request
- the capabilities of the subcluster that
wants to keep alive its participation in federationorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic GetSubClusterInfoResponse getSubCluster(GetSubClusterInfoRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStore
SubClusterId
. The membership information includes the cluster
endpoint and current capabilities as represented by SubClusterInfo
.getSubCluster
in interface FederationMembershipStateStore
request
- the subcluster whose information is requiredSubClusterInfo
, or null
if there is no mapping
for the subclusterorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic GetSubClustersInfoResponse getSubClusters(GetSubClustersInfoRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationMembershipStateStore
SubClusterInfo
.getSubClusters
in interface FederationMembershipStateStore
request
- request for sub-clusters informationSubClusterInfo
keyed by the SubClusterId
org.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic GetSubClusterPolicyConfigurationResponse getPolicyConfiguration(GetSubClusterPolicyConfigurationRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStore
getPolicyConfiguration
in interface FederationPolicyStore
request
- the queue whose SubClusterPolicyConfiguration
is
requiredSubClusterPolicyConfiguration
for the specified queue,
or null
if there is no mapping for the queueorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic SetSubClusterPolicyConfigurationResponse setPolicyConfiguration(SetSubClusterPolicyConfigurationRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStore
setPolicyConfiguration
in interface FederationPolicyStore
request
- the SubClusterPolicyConfiguration
with the
corresponding queueSubClusterPolicyConfiguration
for the specified queueorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic GetSubClusterPoliciesConfigurationsResponse getPoliciesConfigurations(GetSubClusterPoliciesConfigurationsRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStore
getPoliciesConfigurations
in interface FederationPolicyStore
request
- empty to represent all configured queues in the systemorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic DeleteSubClusterPoliciesConfigurationsResponse deletePoliciesConfigurations(DeleteSubClusterPoliciesConfigurationsRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationPolicyStore
deletePoliciesConfigurations
in interface FederationPolicyStore
request
- List containing delete queues.org.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic DeletePoliciesConfigurationsResponse deleteAllPoliciesConfigurations(DeletePoliciesConfigurationsRequest request) throws Exception
FederationPolicyStore
deleteAllPoliciesConfigurations
in interface FederationPolicyStore
request
- delete request.Exception
- if the request is invalid/failspublic Version getCurrentVersion()
FederationStateStore
Version
of the underlying federation state store client.getCurrentVersion
in interface FederationStateStore
Version
of the underlying federation store clientpublic Version loadVersion() throws Exception
FederationStateStore
loadVersion
in interface FederationStateStore
Version
of the federation state storeException
- an exception occurred in load version.public void storeVersion() throws Exception
FederationStateStore
storeVersion
in interface FederationStateStore
Exception
- an exception occurred in store version.public void deleteStateStore() throws Exception
FederationStateStore
deleteStateStore
in interface FederationStateStore
Exception
- an exception occurred in delete store.protected void storeApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, byte[] data) throws Exception
Exception
protected void updateApplicationStateInternal(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, byte[] data) throws Exception
Exception
@VisibleForTesting public ZKFederationStateStoreOpDurations getOpDurations()
public AddReservationHomeSubClusterResponse addReservationHomeSubCluster(AddReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStore
SubClusterId
of the newly submitted
ReservationId
. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure. If a
mapping for the Reservation already existed, the SubClusterId
in
this response will return the existing mapping which might be different
from that in the AddReservationHomeSubClusterRequest
.addReservationHomeSubCluster
in interface FederationReservationHomeSubClusterStore
request
- the request to register a new Reservation with its home
sub-clusterAddReservationHomeSubClusterRequest
containing the home
sub-cluster of the Reservation. Otherwise, an exception reporting
reason for a failureorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic GetReservationHomeSubClusterResponse getReservationHomeSubCluster(GetReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStore
ReservationId
.getReservationHomeSubCluster
in interface FederationReservationHomeSubClusterStore
request
- contains the Reservation queriedReservationHomeSubCluster
containing the Reservation's home
subclusterorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic GetReservationsHomeSubClusterResponse getReservationsHomeSubCluster(GetReservationsHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStore
ReservationHomeSubCluster
list representing the mapping of
all submitted Reservations to it's home sub-cluster.getReservationsHomeSubCluster
in interface FederationReservationHomeSubClusterStore
request
- empty representing all Reservationsorg.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic DeleteReservationHomeSubClusterResponse deleteReservationHomeSubCluster(DeleteReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStore
SubClusterId
of a previously submitted
ReservationId
. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.deleteReservationHomeSubCluster
in interface FederationReservationHomeSubClusterStore
request
- the request to delete the home sub-cluster of a reservation.org.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic UpdateReservationHomeSubClusterResponse updateReservationHomeSubCluster(UpdateReservationHomeSubClusterRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException
FederationReservationHomeSubClusterStore
SubClusterId
of a previously submitted
ReservationId
. Currently response is empty if the operation was
successful, if not an exception reporting reason for a failure.updateReservationHomeSubCluster
in interface FederationReservationHomeSubClusterStore
request
- the request to update the home sub-cluster of a reservation.org.apache.hadoop.yarn.exceptions.YarnException
- if the request is invalid/failspublic RouterMasterKeyResponse storeNewMasterKey(RouterMasterKeyRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
storeNewMasterKey
in interface FederationDelegationTokenStateStore
request
- The request contains RouterMasterKey, which is an abstraction for DelegationKeyorg.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessful.IOException
- An IO Error occurred.public RouterMasterKeyResponse removeStoredMasterKey(RouterMasterKeyRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
removeStoredMasterKey
in interface FederationDelegationTokenStateStore
request
- The request contains RouterMasterKey, which is an abstraction for DelegationKeyorg.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessful.IOException
- An IO Error occurred.public RouterMasterKeyResponse getMasterKeyByDelegationKey(RouterMasterKeyRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
getMasterKeyByDelegationKey
in interface FederationDelegationTokenStateStore
request
- The request contains RouterMasterKey, which is an abstraction for DelegationKeyorg.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessful.IOException
- An IO Error occurred.public RouterRMTokenResponse storeNewToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
storeNewToken
in interface FederationDelegationTokenStateStore
request
- The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessful.IOException
- An IO Error occurred.public RouterRMTokenResponse updateStoredToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
updateStoredToken
in interface FederationDelegationTokenStateStore
request
- The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessful.IOException
- An IO Error occurred.public RouterRMTokenResponse removeStoredToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
removeStoredToken
in interface FederationDelegationTokenStateStore
request
- The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessful.IOException
- An IO Error occurred.public RouterRMTokenResponse getTokenByRouterStoreToken(RouterRMTokenRequest request) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
getTokenByRouterStoreToken
in interface FederationDelegationTokenStateStore
request
- The request contains RouterRMToken (RMDelegationTokenIdentifier and renewDate)org.apache.hadoop.yarn.exceptions.YarnException
- if the call to the state store is unsuccessfulIOException
- An IO Error occurredpublic int incrementDelegationTokenSeqNum()
incrementDelegationTokenSeqNum
in interface FederationDelegationTokenStateStore
public int getDelegationTokenSeqNum()
getDelegationTokenSeqNum
in interface FederationDelegationTokenStateStore
public void setDelegationTokenSeqNum(int seqNum)
setDelegationTokenSeqNum
in interface FederationDelegationTokenStateStore
seqNum
- sequenceNum.public int getCurrentKeyId()
getCurrentKeyId
in interface FederationDelegationTokenStateStore
public int incrementCurrentKeyId()
incrementCurrentKeyId
in interface FederationDelegationTokenStateStore
Copyright © 2008–2024 Apache Software Foundation. All rights reserved.