[Flink] Slot SlotPool related to Flink resources

Posted by gerkintrigg on Sat, 22 Jan 2022 03:32:25 +0100

1. General

Reprint and supplement: http://www.qishunwang.net/news_show_82511.aspx

2.SlotPool

2.1. introduce

slot pool is the pool used by JobMaster to manage slots Is an interface class that defines the management operations of related slots


The main methods are as follows

2.1.1. Life cycle related interfaces

Interfacemeaning
startstart-up
suspendHang
closeclose

2.1.2 resource manager connection

Interfacemeaning
connectToResourceManagerEstablish a connection with ResourceManager
disconnectResourceManagerClose ResourceManager connection
registerTaskManagerRegister a TaskExecutor with the given ResourceId
releaseTaskManagerRelease TaskExecutor

2.1.3 relevant to slot operation

Interfacemeaning
offerSlotsRelease slot
failAllocationIdentify the slot as failed according to the given allocation id
getAvailableSlotsInformationGet the currently available slots information
getAllocatedSlotsInformationGet all slot information
allocateAvailableSlotAllocate available slot s with the given allocation id under the given request id.
If no slot with the given allocation id is available, this method returns {@ code null}.
requestNewAllocatedSlotRequest the allocation of a new slot from the resource manager.
This method does not return a slot from a slot that is already available in the pool, but adds a new slot to the pool, which will be allocated and returned immediately.
requestNewAllocatedBatchSlotRequest allocation of a new batch slot from the resource manager
Unlike ordinary slots, batch slots will time out only when the slot pool does not contain an appropriate slot.
In addition, it does not respond to fault signals from the resource manager.
disableBatchSlotRequestTimeoutCheckDisable batch slot request timeout check.
Called when someone else wants to take over the timeout check responsibility.
createAllocatedSlotReportCreates a report on assigned slot s belonging to the specified task manager.

3.SlotPoolImpl implementation class

SlotPoolImpl is the implementation class of the SlotPool interface

The slot pool serves slot requests issued by {@ link ExecutionGraph}.

When it cannot provide a slot request, it will try to get a new slot from the resource manager.

If no ResourceManager is currently available, or if the ResourceManager rejects it, or if the request times out, it will fail the slot request.

The slot pool also saves all the slots provided to it and accepted, so registered free slots can be provided even if the resource manager is closed.

Slots are released only when they are useless, for example, when the job is fully running, but we still have some available slot s.

All allocations or slot offers will be identified by their own generated AllocationID, which we will use to disambiguate.

3.1. attribute

	/** The interval (in milliseconds) in which the SlotPool writes its slot distribution on debug level.
	 * SlotPool The time interval (in milliseconds) at the debug level to write its slot distribution.
	 * */
	private static final long STATUS_LOG_INTERVAL_MS = 60_000;

	private final JobID jobId;

	/** All registered TaskManagers, slots will be accepted and used only if the resource is registered.
	 * All registered taskmanagers and slot s will be accepted and used only when the resource is registered.
	 * */
	private final HashSet<ResourceID> registeredTaskManagers;

	/** The book-keeping of all allocated slots.
	 * //All slots assigned to the current JobManager
	 * */
	private final AllocatedSlots allocatedSlots;

	/** The book-keeping of all available slots.
	 * All available slots (already assigned to the JobManager, but the payload has not been loaded)
	 * */
	private final AvailableSlots availableSlots;

	/** All pending requests waiting for slots.
	 * All slot request s in the waiting state (requests have been sent to the resource manager) wait for all pending requests from slots.
	 * */
	private final DualKeyLinkedMap<SlotRequestId, AllocationID, PendingRequest> pendingRequests;

	/** The requests that are waiting for the resource manager to be connected.
	 * slot request in waiting status (the request has not been sent to the ResourceManager, and no connection has been established with the ResourceManager at this time)
	 * Wait for a request to connect to the resource manager.
	 * */
	private final LinkedHashMap<SlotRequestId, PendingRequest> waitingForResourceManager;

	/** Timeout for external request calls (e.g. to the ResourceManager or the TaskExecutor).
	 * External request invocation timed out (for example, to ResourceManager or TaskExecutor).
	 * */
	private final Time rpcTimeout;

	/** Timeout for releasing idle slots.
	 * Free slots timeout
	 * */
	private final Time idleSlotTimeout;

	/** Timeout for batch slot requests.
	 * Batch slot request timed out
	 * */
	private final Time batchSlotTimeout;

	private final Clock clock;

	/** the fencing token of the job manager. */
	private JobMasterId jobMasterId;

	/** The gateway to communicate with resource manager. */
	private ResourceManagerGateway resourceManagerGateway;

	private String jobManagerAddress;

	// Component main thread executor
	private ComponentMainThreadExecutor componentMainThreadExecutor;

3.2 life cycle related interfaces

Interfacemeaning
startstart-up
suspendHang
closeclose

3.2.1 start method

/**
	 * Start the slot pool to accept RPC calls.
	 *
	 * Start the slot pool to accept RPC calls.
	 *
	 * @param jobMasterId The necessary leader id for running the job.
	 * @param newJobManagerAddress for the slot requests which are sent to the resource manager
	 * @param componentMainThreadExecutor The main thread executor for the job master's main thread.
	 */
	public void start(
		@Nonnull JobMasterId jobMasterId,
		@Nonnull String newJobManagerAddress,
		@Nonnull ComponentMainThreadExecutor componentMainThreadExecutor) throws Exception {

		this.jobMasterId = jobMasterId;
		this.jobManagerAddress = newJobManagerAddress;
		this.componentMainThreadExecutor = componentMainThreadExecutor;

		// Timeout related operations
		scheduleRunAsync(this::checkIdleSlot, idleSlotTimeout);
		scheduleRunAsync(this::checkBatchSlotTimeout, batchSlotTimeout);

		if (log.isDebugEnabled()) {
			scheduleRunAsync(this::scheduledLogStatus, STATUS_LOG_INTERVAL_MS, TimeUnit.MILLISECONDS);
		}
	}

3.2.2 suspend

/**
	 * Suspends this pool, meaning it has lost its authority to accept and distribute slots.
	 *
	 * Suspending this pool means that it has lost its permission to accept and distribute slot s.
	 */
	@Override
	public void suspend() {

		componentMainThreadExecutor.assertRunningInMainThread();

		log.info("Suspending SlotPool.");

		// cancel all pending allocations --> we can request these slots
		// again after we regained the leadership
		Set<AllocationID> allocationIds = pendingRequests.keySetB();

		for (AllocationID allocationId : allocationIds) {
			// resourceManagerGateway cancels the SlotRequest operation
			resourceManagerGateway.cancelSlotRequest(allocationId);
		}

		// do not accept any requests
		jobMasterId = null;
		resourceManagerGateway = null;

		// Clear (but not release!) the available slots. The TaskManagers should re-register them
		// at the new leader JobManager/SlotPool
		clear();
	}

3.2.3 close

@Override
	public void close() {
		log.info("Stopping SlotPool.");
		// cancel all pending allocations
		// Cancel pending SlotRequests
		Set<AllocationID> allocationIds = pendingRequests.keySetB();

		for (AllocationID allocationId : allocationIds) {
			resourceManagerGateway.cancelSlotRequest(allocationId);
		}

		// Release resources release all registered slots by releasing the corresponding TaskExecutor
		// release all registered slots by releasing the corresponding TaskExecutors
		for (ResourceID taskManagerResourceId : registeredTaskManagers) {
			final FlinkException cause = new FlinkException(
				"Releasing TaskManager " + taskManagerResourceId + ", because of stopping of SlotPool");
			releaseTaskManagerInternal(taskManagerResourceId, cause);
		}

		clear();
	}

3.3 resource manager connection

Interfacemeaning
connectToResourceManagerEstablish connection with ResourceManager
disconnectResourceManagerClose the ResourceManager connection
registerTaskManagerRegister a TaskExecutor with the given ResourceId
releaseTaskManagerRelease TaskExecutor

3.3.1 connectToResourceManager

/**
	 * Establish a connection with ResourceManager to handle blocked / pending requests
	 * @param resourceManagerGateway  The RPC gateway for the resource manager.
	 */
	@Override
	public void connectToResourceManager(@Nonnull ResourceManagerGateway resourceManagerGateway) {
		this.resourceManagerGateway = checkNotNull(resourceManagerGateway);

		// Processing pending PendingRequest requests
		// work on all slots waiting for this connection
		for (PendingRequest pendingRequest : waitingForResourceManager.values()) {
			// Request RM / get resource
			requestSlotFromResourceManager(resourceManagerGateway, pendingRequest);
		}

		// all sent off
		waitingForResourceManager.clear();
	}

3.3.2 disconnectResourceManager

Close the ResourceManager connection

@Override
	public void disconnectResourceManager() {
		this.resourceManagerGateway = null;
	}

3.3.3 registerTaskManager

/**
	 * Register TaskManager to this pool, only those slots come from registered TaskManager will be considered valid.
	 * Also it provides a way for us to keep "dead" or "abnormal" TaskManagers out of this pool.
	 *
	 * @param resourceID The id of the TaskManager
	 *
	 *
	 *                      If you register a TaskManager with this pool, only slot s from a registered TaskManager will be considered valid.
	 * It also provides us with a way to keep "dead" or "abnormal" task managers away from the pool
	 */
	@Override
	public boolean registerTaskManager(final ResourceID resourceID) {

		componentMainThreadExecutor.assertRunningInMainThread();

		log.debug("Register new TaskExecutor {}.", resourceID);
		return registeredTaskManagers.add(resourceID);
	}

3.3.4 releaseTaskManager

/**
	 * Unregister TaskManager from this pool, all the related slots will be released and tasks be canceled. Called
	 * when we find some TaskManager becomes "dead" or "abnormal", and we decide to not using slots from it anymore.

	 * Unregistering the TaskManager from the pool releases all relevant slot s and cancels the task.
	 * Called when we find that a task manager becomes "dead" or "abnormal" and we decide not to use the slot in it anymore.
	 * 
	 * @param resourceId The id of the TaskManager
	 * @param cause for the releasing of the TaskManager
	 */
	@Override
	public boolean releaseTaskManager(final ResourceID resourceId, final Exception cause) {

		componentMainThreadExecutor.assertRunningInMainThread();

		if (registeredTaskManagers.remove(resourceId)) {
			releaseTaskManagerInternal(resourceId, cause);
			return true;
		} else {
			return false;
		}
	}

3.4 relevant to slot operation

Interfacemeaning
offerSlotsConsumption slot
failAllocationIdentify the slot as failed according to the given allocation id
getAvailableSlotsInformationGet the currently available slots information
getAllocatedSlotsInformationGet all slot information
allocateAvailableSlotAllocate available slot s with the given allocation id under the given request id.
If no slot with the given allocation id is available, this method returns {@ code null}.
requestNewAllocatedSlotRequest the allocation of a new slot from the resource manager.
This method does not return a slot from a slot that is already available in the pool, but adds a new slot to the pool, which will be allocated and returned immediately.
requestNewAllocatedBatchSlotRequest allocation of a new batch slot from the resource manager
Unlike ordinary slots, batch slots will time out only when the slot pool does not contain an appropriate slot.
In addition, it does not respond to fault signals from the resource manager.
disableBatchSlotRequestTimeoutCheckDisable batch slot request timeout check.
Called when someone else wants to take over the timeout check responsibility.
createAllocatedSlotReportCreates a report on assigned slot s belonging to the specified task manager.

3.4.1 offerSlots

/**
	 * According to the allocationid, taskexecutor provides Slot
	 *
	 * AllocationID It is initially generated by the pool and transferred to task manager through resource manager
	 *
	 * We use it to distinguish the different distributions we issue.
	 *
	 * If we find that a Slot does not match or do not actually wait for the pending request of this Slot (which may be completed by other returned slots), the Slot offer may be rejected.
	 *
	 * Slot offering by TaskExecutor with AllocationID. The AllocationID is originally generated by this pool and
	 * transfer through the ResourceManager to TaskManager. We use it to distinguish the different allocation
	 * we issued. Slot offering may be rejected if we find something mismatching or there is actually no pending
	 * request waiting for this slot (maybe fulfilled by some other returned slot).
	 *
	 * @param taskManagerLocation location from where the offer comes from
	 * @param taskManagerGateway TaskManager gateway
	 * @param slotOffer the offered slot
	 * @return True if we accept the offering
	 */
	boolean offerSlot(
			final TaskManagerLocation taskManagerLocation,
			final TaskManagerGateway taskManagerGateway,
			final SlotOffer slotOffer) {

		componentMainThreadExecutor.assertRunningInMainThread();

		// Check whether the TaskManager is valid
		// check if this TaskManager is valid
		final ResourceID resourceID = taskManagerLocation.getResourceID();
		final AllocationID allocationID = slotOffer.getAllocationId();

		// Must be a slotOffer in a registered taskmanager
		if (!registeredTaskManagers.contains(resourceID)) {
			log.debug("Received outdated slot offering [{}] from unregistered TaskManager: {}",
					slotOffer.getAllocationId(), taskManagerLocation);
			return false;
		}

		// If the AllocationID associated with the current slot already appears in the SlotPool, check whether this slot has been used
		// check whether we have already using this slot
		AllocatedSlot existingSlot;
		if ((existingSlot = allocatedSlots.get(allocationID)) != null ||
			(existingSlot = availableSlots.get(allocationID)) != null) {

			// we need to figure out if this is a repeated offer for the exact same slot,
			// or another offer that comes from a different TaskManager after the ResourceManager
			// re-tried the request

			//  We need to find out that this is a repeated offer for exactly the same slot,
			//  Or another offer from a different task manager after the resource manager retries the request

			// We write this by comparing the slot ID, because the slot ID is the identifier of the actual slot on the task manager

			// we write this in terms of comparing slot IDs, because the Slot IDs are the identifiers of
			// the actual slots on the TaskManagers
			// Note: The slotOffer should have the SlotID

			// Get existing SlotID
			final SlotID existingSlotId = existingSlot.getSlotId();
			// Get new SlotID
			final SlotID newSlotId = new SlotID(taskManagerLocation.getResourceID(), slotOffer.getSlotIndex());

			//This slot has been accepted by the SlotPool before, which is equivalent to the TaskExecutor sending a duplicate offer
			if (existingSlotId.equals(newSlotId)) {
				log.info("Received repeated offer for slot [{}]. Ignoring.", allocationID);

				// return true here so that the sender will get a positive acknowledgement to the retry
				// and mark the offering as a success
				return true;
			} else {
				//There is already another AllocatedSlot associated with this AllocationID, so the current slot cannot be accepted
				// the allocation has been fulfilled by another slot, reject the offer so the task executor
				// will offer the slot to the resource manager
				return false;
			}
		}

		// This means that this slot has not been used yet
		//The allocation ID associated with this slot has not appeared before

		//Create an AllocatedSlot object to represent the newly allocated slot
		final AllocatedSlot allocatedSlot = new AllocatedSlot(
			allocationID,
			taskManagerLocation,
			slotOffer.getSlotIndex(),
			slotOffer.getResourceProfile(),
			taskManagerGateway);

		// Check whether there is a request associated with this AllocationID
		// check whether we have request waiting for this slot
		PendingRequest pendingRequest = pendingRequests.removeKeyB(allocationID);
		if (pendingRequest != null) {
			// we were waiting for this!
			//There is a pending request waiting for this slot
			allocatedSlots.add(pendingRequest.getSlotRequestId(), allocatedSlot);

			//Try to complete the waiting request
			if (!pendingRequest.getAllocatedSlotFuture().complete(allocatedSlot)) {
				// we could not complete the pending slot future --> try to fulfill another pending request
				//failed
				allocatedSlots.remove(pendingRequest.getSlotRequestId());
				//Try to satisfy other waiting requests, and use slot to complete pending requests in the order of requests
				tryFulfillSlotRequestOrMakeAvailable(allocatedSlot);
			} else {
				log.debug("Fulfilled slot request [{}] with allocated slot [{}].", pendingRequest.getSlotRequestId(), allocationID);
			}
		}
		else {
			//There is no request waiting for this slot. The request may have been satisfied
			// we were actually not waiting for this:
			//   - could be that this request had been fulfilled
			//   - we are receiving the slots from TaskManagers after becoming leaders
			//Try to satisfy other waiting requests
			tryFulfillSlotRequestOrMakeAvailable(allocatedSlot);
		}

		// we accepted the request in any case. slot will be released after it idled for
		// too long and timed out
		// We accepted the request anyway
		// The slot will be released after too long idle time and timeout
		return true;
	}

tryFulfillSlotRequestOrMakeAvailable

/**
	 * Tries to fulfill with the given allocated slot a pending slot request or add the
	 * allocated slot to the set of available slots if no matching request is available.
	 *
	 * Attempting to complete a pending slot request with the given allocated slot,
	 * Or if there is no matching request, return the allocated slot to the available slot set.
	 *
	 * @param allocatedSlot which shall be returned
	 */
	private void tryFulfillSlotRequestOrMakeAvailable(AllocatedSlot allocatedSlot) {
		Preconditions.checkState(!allocatedSlot.isUsed(), "Provided slot is still in use.");

		//Find the pending requests that match the computing resources of the current AllocatedSlot
		final PendingRequest pendingRequest = pollMatchingPendingRequest(allocatedSlot);

		if (pendingRequest != null) {
			//If there are matching requests, AllocatedSlot is assigned to the waiting requests
			log.debug("Fulfilling pending slot request [{}] early with returned slot [{}]",
				pendingRequest.getSlotRequestId(), allocatedSlot.getAllocationId());

			// Add the currently allocated slots to the allocated allocated slots collection to identify that they have been used
			allocatedSlots.add(pendingRequest.getSlotRequestId(), allocatedSlot);

			// Callback request and return allocatedSlot information Identity slot allocation completed
			pendingRequest.getAllocatedSlotFuture().complete(allocatedSlot);
		} else {
			//If not, the allocated slot becomes available
			// There is no PendingRequest available. Return allocatedSlot
			log.debug("Adding returned slot [{}] to available slots", allocatedSlot.getAllocationId());
			availableSlots.add(allocatedSlot, clock.relativeTimeMillis());
		}
	}

3.4.2 failAllocation

@Override
	public Optional<ResourceID> failAllocation(final AllocationID allocationID, final Exception cause) {

		componentMainThreadExecutor.assertRunningInMainThread();

		// Get PendingRequest
		final PendingRequest pendingRequest = pendingRequests.removeKeyB(allocationID);
		if (pendingRequest != null) {
			if (isBatchRequestAndFailureCanBeIgnored(pendingRequest, cause)) {
				// pending batch requests don't react to this signal --> put it back
				pendingRequests.put(pendingRequest.getSlotRequestId(), allocationID, pendingRequest);
			} else {
				// request was still pending
				failPendingRequest(pendingRequest, cause);
			}
			return Optional.empty();
		}
		else {
			return tryFailingAllocatedSlot(allocationID, cause);
		}

		// TODO: add some unit tests when the previous two are ready, the allocation may failed at any phase
	}

tryFailingAllocatedSlot

	private Optional<ResourceID> tryFailingAllocatedSlot(AllocationID allocationID, Exception cause) {
		// Get AllocatedSlot with allocation failure
		AllocatedSlot allocatedSlot = availableSlots.tryRemove(allocationID);

		if (allocatedSlot == null) {
			allocatedSlot = allocatedSlots.remove(allocationID);
		}

		if (allocatedSlot != null) {
			log.debug("Failed allocated slot [{}]: {}", allocationID, cause.getMessage());

			// notify TaskExecutor about the failure
			// Notification TaskExecutor assignment failed
			allocatedSlot.getTaskManagerGateway().freeSlot(allocationID, cause, rpcTimeout);
			// release the slot.
			// since it is not in 'allocatedSlots' any more, it will be dropped o return'
			// Release the slot and discard it
			allocatedSlot.releasePayload(cause);

			final ResourceID taskManagerId = allocatedSlot.getTaskManagerId();

			if (!availableSlots.containsTaskManager(taskManagerId) && !allocatedSlots.containResource(taskManagerId)) {
				return Optional.of(taskManagerId);
			}
		}

		return Optional.empty();
	}

3.4.3 getAvailableSlotsInformation

Get available slot information

/**
	 * Lists the currently available slot s
	 * @return
	 */
	@Override
	@Nonnull
	public Collection<SlotInfoWithUtilization> getAvailableSlotsInformation() {
		final Map<ResourceID, Set<AllocatedSlot>> availableSlotsByTaskManager = availableSlots.getSlotsByTaskManager();
		final Map<ResourceID, Set<AllocatedSlot>> allocatedSlotsSlotsByTaskManager = allocatedSlots.getSlotsByTaskManager();

		return availableSlotsByTaskManager.entrySet().stream()
			.flatMap(entry -> {
				final int numberAllocatedSlots = allocatedSlotsSlotsByTaskManager.getOrDefault(entry.getKey(), Collections.emptySet()).size();
				final int numberAvailableSlots = entry.getValue().size();
				final double taskExecutorUtilization = (double) numberAllocatedSlots / (numberAllocatedSlots + numberAvailableSlots);

				return entry.getValue().stream().map(slot -> SlotInfoWithUtilization.from(slot, taskExecutorUtilization));
			})
			.collect(Collectors.toList());
	}

3.4.4 getAllocatedSlotsInformation

Get all allocated solt information

	private Collection<SlotInfo> getAllocatedSlotsInformation() {
		return allocatedSlots.listSlotInfo();
	}

3.4.5 allocateAvailableSlot

Get all valid solt information

	/**
	 * Assign the slot associated with the allocationID to the request corresponding to the slotRequestId
	 * @param slotRequestId identifying the requested slot
	 * @param allocationID the allocation id of the requested available slot
	 * @return
	 */
	@Override
	public Optional<PhysicalSlot> allocateAvailableSlot(
		@Nonnull SlotRequestId slotRequestId,
		@Nonnull AllocationID allocationID) {

		componentMainThreadExecutor.assertRunningInMainThread();
		//Remove from availableSlots
		AllocatedSlot allocatedSlot = availableSlots.tryRemove(allocationID);
		if (allocatedSlot != null) {
			//Join the assigned mapping relationship
			allocatedSlots.add(slotRequestId, allocatedSlot);
			return Optional.of(allocatedSlot);
		} else {
			return Optional.empty();
		}
	}

3.4.6 requestNewAllocatedSlot

Request the allocation of a new slot from the resource manager. This method does not return a slot from a slot that is already available in the pool, but adds a new slot to the pool, which will be allocated and returned immediately.

/**
	 * Apply to RM for a new slot
	 * 
	 * Request the allocation of a new slot from the resource manager. This method does not return a slot from a slot that is already available in the pool,
	 * Instead, a new slot will be added to the pool, which will be allocated and returned immediately.
	 *
	 * @param slotRequestId identifying the requested slot
	 * @param resourceProfile resource profile that specifies the resource requirements for the requested slot
	 * @param timeout timeout for the allocation procedure
	 * @return
	 */
	@Nonnull
	@Override
	public CompletableFuture<PhysicalSlot> requestNewAllocatedSlot(
			@Nonnull SlotRequestId slotRequestId,
			@Nonnull ResourceProfile resourceProfile,
			Time timeout) {

		componentMainThreadExecutor.assertRunningInMainThread();

		final PendingRequest pendingRequest = PendingRequest.createStreamingRequest(slotRequestId, resourceProfile);

		// register request timeout
		FutureUtils
			.orTimeout(
				pendingRequest.getAllocatedSlotFuture(),
				timeout.toMilliseconds(),
				TimeUnit.MILLISECONDS,
				componentMainThreadExecutor)
			.whenComplete(
				(AllocatedSlot ignored, Throwable throwable) -> {
					if (throwable instanceof TimeoutException) {
						timeoutPendingSlotRequest(slotRequestId);
					}
				});

		return requestNewAllocatedSlotInternal(pendingRequest)
			.thenApply((Function.identity()));
	}

requestNewAllocatedSlotInternal

/**
	 *
	 * Request a new slot from RM
	 *
	 * Requests a new slot from the ResourceManager. If there is currently not ResourceManager
	 * connected, then the request is stashed and send once a new ResourceManager is connected.
	 *
	 * @param pendingRequest pending slot request
	 * @return An {@link AllocatedSlot} future which is completed once the slot is offered to the {@link SlotPool}
	 */
	@Nonnull
	private CompletableFuture<AllocatedSlot> requestNewAllocatedSlotInternal(PendingRequest pendingRequest) {

		if (resourceManagerGateway == null) {
			stashRequestWaitingForResourceManager(pendingRequest);
		} else {
			// Request a new slot from RM
			requestSlotFromResourceManager(resourceManagerGateway, pendingRequest);
		}

		return pendingRequest.getAllocatedSlotFuture();
	}

3.4.7 requestNewAllocatedBatchSlot

A request to allocate a new batch slot from the resource manager is different from an ordinary slot. The batch slot will time out only when the slot pool does not contain an appropriate slot. In addition, it does not respond to fault signals from the resource manager.

@Nonnull
	@Override
	public CompletableFuture<PhysicalSlot> requestNewAllocatedBatchSlot(
		@Nonnull SlotRequestId slotRequestId,
		@Nonnull ResourceProfile resourceProfile) {

		componentMainThreadExecutor.assertRunningInMainThread();

		final PendingRequest pendingRequest = PendingRequest.createBatchRequest(slotRequestId, resourceProfile);

		return requestNewAllocatedSlotInternal(pendingRequest)
			.thenApply(Function.identity());
	}

3.4.8 disableBatchSlotRequestTimeoutCheck

Disable batch slot request timeout check. Called when someone else wants to take over the timeout check responsibility.

  @Override
    public void disableBatchSlotRequestTimeoutCheck() {
        batchSlotRequestTimeoutCheckEnabled = false;
    }

3.4.9 createAllocatedSlotReport

Creates a report on assigned slot s belonging to the specified task manager.

/**
	 * Creates a report on assigned slot s belonging to the specified task manager.
	 * @param taskManagerId identifies the task manager
	 * @return
	 */
	@Override
	public AllocatedSlotReport createAllocatedSlotReport(ResourceID taskManagerId) {
		final Set<AllocatedSlot> availableSlotsForTaskManager = availableSlots.getSlotsForTaskManager(taskManagerId);
		final Set<AllocatedSlot> allocatedSlotsForTaskManager = allocatedSlots.getSlotsForTaskManager(taskManagerId);

		List<AllocatedSlotInfo> allocatedSlotInfos = new ArrayList<>(
				availableSlotsForTaskManager.size() + allocatedSlotsForTaskManager.size());
		for (AllocatedSlot allocatedSlot : Iterables.concat(availableSlotsForTaskManager, allocatedSlotsForTaskManager)) {
			allocatedSlotInfos.add(
					new AllocatedSlotInfo(allocatedSlot.getPhysicalSlotNumber(), allocatedSlot.getAllocationId()));
		}
		return new AllocatedSlotReport(jobId, allocatedSlotInfos);
	}