@InterfaceStability.Stable @InterfaceAudience.Private public interface CSQueue extends SchedulerQueue<CSQueue>
CSQueue
represents a node in the tree of
hierarchical queues in the CapacityScheduler
.Modifier and Type | Method and Description |
---|---|
boolean |
accept(org.apache.hadoop.yarn.api.records.Resource cluster,
ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request) |
void |
apply(org.apache.hadoop.yarn.api.records.Resource cluster,
ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request) |
CSAssignment |
assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource,
CandidateNodeSet<FiCaSchedulerNode> candidates,
ResourceLimits resourceLimits,
SchedulingMode schedulingMode)
Assign containers to applications in the queue or it's children (if any).
|
CSAssignment |
assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource,
FiCaSchedulerNode node,
ResourceLimits resourceLimits,
SchedulingMode schedulingMode) |
void |
attachContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource,
FiCaSchedulerApp application,
RMContainer container)
Attach a container to this queue
|
void |
collectSchedulerApplications(Collection<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> apps)
Adds all applications in the queue and its subqueues to the given collection.
|
void |
completedContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource,
FiCaSchedulerApp application,
FiCaSchedulerNode node,
RMContainer container,
org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus,
RMContainerEventType event,
CSQueue childQueue,
boolean sortQueues)
A container assigned to the queue has completed.
|
void |
decPendingResource(String nodeLabel,
org.apache.hadoop.yarn.api.records.Resource resourceToDec)
When an outstanding resource is fulfilled or canceled, calling this will
decrease pending resource in a queue.
|
void |
decUsedResource(String nodePartition,
org.apache.hadoop.yarn.api.records.Resource resourceToDec,
SchedulerApplicationAttempt application)
When partition of node updated, we will update queue's resource usage if it
has container(s) running on that.
|
void |
detachContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource,
FiCaSchedulerApp application,
RMContainer container)
Detach a container from this queue
|
void |
finishApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
String user)
An application submitted to this queue has finished.
|
void |
finishApplicationAttempt(FiCaSchedulerApp application,
String queue)
An application attempt submitted to this queue has finished.
|
float |
getAbsoluteCapacity()
Get capacity of the parent of the queue as a function of the
cumulative capacity in the cluster.
|
float |
getAbsoluteMaximumCapacity()
Get maximum-capacity of the queue as a funciton of the cumulative capacity
of the cluster.
|
float |
getAbsoluteUsedCapacity()
Get the current absolute used capacity of the queue
relative to the entire cluster.
|
AbstractUsersManager |
getAbstractUsersManager()
Get the
AbstractUsersManager for the queue. |
float |
getCapacity()
Get the configured capacity of the queue.
|
AbstractCSQueue.CapacityConfigType |
getCapacityConfigType()
Get CapacityConfigType as PERCENTAGE or ABSOLUTE_RESOURCE.
|
List<CSQueue> |
getChildQueues()
Get child queues
|
long |
getDefaultApplicationLifetime()
Get the default lifetime in seconds of an application which is submitted to
this queue.
|
boolean |
getDefaultAppLifetimeWasSpecifiedInConfig()
Get the indicator of whether or not the default application lifetime was
set by a config property or was calculated by the capacity scheduler.
|
org.apache.hadoop.yarn.api.records.Resource |
getEffectiveCapacity(String label)
Get effective capacity of queue.
|
org.apache.hadoop.yarn.api.records.Resource |
getEffectiveCapacityDown(String label,
org.apache.hadoop.yarn.api.records.Resource factor)
Get effective capacity of queue.
|
org.apache.hadoop.yarn.api.records.Resource |
getEffectiveMaxCapacity(String label)
Get effective max capacity of queue.
|
org.apache.hadoop.yarn.api.records.Resource |
getEffectiveMaxCapacityDown(String label,
org.apache.hadoop.yarn.api.records.Resource factor)
Get effective max capacity of queue.
|
boolean |
getIntraQueuePreemptionDisabled()
Check whether intra-queue preemption is disabled for this queue
|
boolean |
getIntraQueuePreemptionDisabledInHierarchy()
Determines whether or not the intra-queue preemption disabled switch is set
at any level in this queue's hierarchy.
|
org.apache.hadoop.yarn.api.records.Resource |
getMaximumAllocation() |
long |
getMaximumApplicationLifetime()
Get the maximum lifetime in seconds of an application which is submitted to
this queue.
|
float |
getMaximumCapacity()
Get the configured maximum-capacity of the queue.
|
org.apache.hadoop.yarn.api.records.Resource |
getMinimumAllocation() |
String |
getMultiNodeSortingPolicyName()
Get Multi Node scheduling policy name.
|
Set<String> |
getNodeLabelsForQueue()
Get valid Node Labels for this queue
|
int |
getNumApplications()
Get the number of applications in the queue.
|
CSQueue |
getParent()
Get the parent
Queue . |
boolean |
getPreemptionDisabled()
Check whether disable_preemption property is set for this queue
|
org.apache.hadoop.yarn.api.records.Priority |
getPriority()
Get priority of queue
|
org.apache.hadoop.yarn.security.PrivilegedEntity |
getPrivilegedEntity() |
QueueCapacities |
getQueueCapacities()
Get QueueCapacities of this queue
|
String |
getQueueName()
Get the queue's internal reference name.
|
String |
getQueuePath()
Get the full name of the queue, including the heirarchy.
|
QueueResourceQuotas |
getQueueResourceQuotas()
Get QueueResourceQuotas associated with each queue.
|
ResourceUsage |
getQueueResourceUsage()
Get ResourceUsage of this queue
|
String |
getQueueShortName()
Get the queue's legacy name.
|
ReentrantReadWriteLock.ReadLock |
getReadLock()
Get readLock associated with the Queue.
|
org.apache.hadoop.yarn.api.records.QueueState |
getState()
Get the current run-state of the queue
|
float |
getUsedCapacity()
Get the current used capacity of nodes without label(s) of the queue
and it's children (if any).
|
org.apache.hadoop.yarn.api.records.Resource |
getUsedResources()
Get the currently utilized resources which allocated at nodes without any
labels in the cluster by the queue and children (if any).
|
Map<String,Float> |
getUserWeights()
Get a map of usernames and weights
|
boolean |
hasAccess(org.apache.hadoop.yarn.api.records.QueueACL acl,
org.apache.hadoop.security.UserGroupInformation user)
Check if the
user has permission to perform the operation |
void |
incUsedResource(String nodePartition,
org.apache.hadoop.yarn.api.records.Resource resourceToInc,
SchedulerApplicationAttempt application)
When partition of node updated, we will update queue's resource usage if it
has container(s) running on that.
|
void |
reinitialize(CSQueue newlyParsedQueue,
org.apache.hadoop.yarn.api.records.Resource clusterResource)
Reinitialize the queue.
|
void |
setParent(CSQueue newParentQueue)
Set the parent
Queue . |
void |
submitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
String user,
String queue)
Submit a new application to the queue.
|
void |
submitApplicationAttempt(FiCaSchedulerApp application,
String userName)
Submit an application attempt to the queue.
|
void |
submitApplicationAttempt(FiCaSchedulerApp application,
String userName,
boolean isMoveApp)
Submit an application attempt to the queue.
|
void |
updateClusterResource(org.apache.hadoop.yarn.api.records.Resource clusterResource,
ResourceLimits resourceLimits)
Update the cluster resource for queues as we add/remove nodes
|
void |
validateSubmitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
String userName,
String queue)
Validate submitApplication api so that moveApplication do a pre-check.
|
activeQueue, stopQueue, updateQueueState
decReservedResource, getAccessibleNodeLabels, getDefaultApplicationPriority, getDefaultNodeLabelExpression, getMetrics, getQueueInfo, getQueueUserAclInfo, incPendingResource, incReservedResource, recoverContainer
CSQueue getParent()
Queue
.getParent
in interface SchedulerQueue<CSQueue>
void setParent(CSQueue newParentQueue)
Queue
.newParentQueue
- new parent queueString getQueueName()
getQueueName
in interface Queue
String getQueueShortName()
String getQueuePath()
org.apache.hadoop.yarn.security.PrivilegedEntity getPrivilegedEntity()
org.apache.hadoop.yarn.api.records.Resource getMaximumAllocation()
org.apache.hadoop.yarn.api.records.Resource getMinimumAllocation()
float getCapacity()
float getAbsoluteCapacity()
float getMaximumCapacity()
float getAbsoluteMaximumCapacity()
float getAbsoluteUsedCapacity()
float getUsedCapacity()
org.apache.hadoop.yarn.api.records.Resource getUsedResources()
org.apache.hadoop.yarn.api.records.QueueState getState()
getState
in interface SchedulerQueue<CSQueue>
List<CSQueue> getChildQueues()
getChildQueues
in interface SchedulerQueue<CSQueue>
boolean hasAccess(org.apache.hadoop.yarn.api.records.QueueACL acl, org.apache.hadoop.security.UserGroupInformation user)
user
has permission to perform the operationvoid submitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String user, String queue) throws org.apache.hadoop.security.AccessControlException
applicationId
- the applicationId of the application being submitteduser
- user who submitted the applicationqueue
- queue to which the application is submittedorg.apache.hadoop.security.AccessControlException
- if any acl violation is there.void submitApplicationAttempt(FiCaSchedulerApp application, String userName)
application
- application whose attempt is being submitted.userName
- userName who submitted the application.void submitApplicationAttempt(FiCaSchedulerApp application, String userName, boolean isMoveApp)
application
- application whose attempt is being submitteduserName
- user who submitted the application attemptisMoveApp
- is application being moved across the queuevoid finishApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String user)
applicationId
- applicationId.user
- user who submitted the applicationvoid finishApplicationAttempt(FiCaSchedulerApp application, String queue)
application
- application attempt.queue
- queue.CSAssignment assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource, CandidateNodeSet<FiCaSchedulerNode> candidates, ResourceLimits resourceLimits, SchedulingMode schedulingMode)
clusterResource
- the resource of the cluster.candidates
- CandidateNodeSet
the nodes that are considered
for the current placement.resourceLimits
- how much overall resource of this queue can use.schedulingMode
- Type of exclusive check when assign container on a
NodeManager, see SchedulingMode
.void completedContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerApp application, FiCaSchedulerNode node, RMContainer container, org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus, RMContainerEventType event, CSQueue childQueue, boolean sortQueues)
clusterResource
- the resource of the clusterapplication
- application to which the container was assignednode
- node on which the container completedcontainer
- completed container,
null
if it was just a reservationcontainerStatus
- ContainerStatus
for the completed
containerchildQueue
- CSQueue
to reinsert in childQueuesevent
- event to be sent to the containersortQueues
- indicates whether it should re-sort the queuesint getNumApplications()
void reinitialize(CSQueue newlyParsedQueue, org.apache.hadoop.yarn.api.records.Resource clusterResource) throws IOException
newlyParsedQueue
- new queue to re-initalize fromclusterResource
- resources in the clusterIOException
- an I/O exception has occurred.void updateClusterResource(org.apache.hadoop.yarn.api.records.Resource clusterResource, ResourceLimits resourceLimits)
clusterResource
- the current cluster resourceresourceLimits
- the current ResourceLimitsAbstractUsersManager getAbstractUsersManager()
AbstractUsersManager
for the queue.getAbstractUsersManager
in interface Queue
AbstractUsersManager
for the queuevoid collectSchedulerApplications(Collection<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> apps)
apps
- the collection to add the applications tovoid detachContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerApp application, RMContainer container)
clusterResource
- the current cluster resourceapplication
- application to which the container was assignedcontainer
- the container to detachvoid attachContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerApp application, RMContainer container)
clusterResource
- the current cluster resourceapplication
- application to which the container was assignedcontainer
- the container to attachboolean getPreemptionDisabled()
boolean getIntraQueuePreemptionDisabled()
boolean getIntraQueuePreemptionDisabledInHierarchy()
QueueCapacities getQueueCapacities()
ResourceUsage getQueueResourceUsage()
void incUsedResource(String nodePartition, org.apache.hadoop.yarn.api.records.Resource resourceToInc, SchedulerApplicationAttempt application)
nodePartition
- node label.resourceToInc
- resource.application
- application.void decUsedResource(String nodePartition, org.apache.hadoop.yarn.api.records.Resource resourceToDec, SchedulerApplicationAttempt application)
nodePartition
- node label.resourceToDec
- resource.application
- application.void decPendingResource(String nodeLabel, org.apache.hadoop.yarn.api.records.Resource resourceToDec)
decPendingResource
in interface Queue
nodeLabel
- asked by applicationresourceToDec
- new resource askedSet<String> getNodeLabelsForQueue()
CSAssignment assignContainers(org.apache.hadoop.yarn.api.records.Resource clusterResource, FiCaSchedulerNode node, ResourceLimits resourceLimits, SchedulingMode schedulingMode)
boolean accept(org.apache.hadoop.yarn.api.records.Resource cluster, ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request)
void apply(org.apache.hadoop.yarn.api.records.Resource cluster, ResourceCommitRequest<FiCaSchedulerApp,FiCaSchedulerNode> request)
ReentrantReadWriteLock.ReadLock getReadLock()
void validateSubmitApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String userName, String queue) throws org.apache.hadoop.security.AccessControlException
applicationId
- Application IDuserName
- User Namequeue
- Queue Nameorg.apache.hadoop.security.AccessControlException
- if any acl violation is there.org.apache.hadoop.yarn.api.records.Priority getPriority()
Map<String,Float> getUserWeights()
QueueResourceQuotas getQueueResourceQuotas()
AbstractCSQueue.CapacityConfigType getCapacityConfigType()
org.apache.hadoop.yarn.api.records.Resource getEffectiveCapacity(String label)
label
- partitionorg.apache.hadoop.yarn.api.records.Resource getEffectiveCapacityDown(String label, org.apache.hadoop.yarn.api.records.Resource factor)
label
- partitionfactor
- factor to normalize downorg.apache.hadoop.yarn.api.records.Resource getEffectiveMaxCapacity(String label)
label
- partitionorg.apache.hadoop.yarn.api.records.Resource getEffectiveMaxCapacityDown(String label, org.apache.hadoop.yarn.api.records.Resource factor)
label
- partitionfactor
- factor to normalize downString getMultiNodeSortingPolicyName()
long getMaximumApplicationLifetime()
long getDefaultApplicationLifetime()
boolean getDefaultAppLifetimeWasSpecifiedInConfig()
Copyright © 2008–2023 Apache Software Foundation. All rights reserved.