WatchService watching files on distributed filesystem from multiple machines

36
June 22, 2022, at 08:20 AM

I am trying to implement a WatchService that watches a filesystem for changes. Once files get created I want to create tasks for each new file and add that task to a distributed queue to let it get processed from multiple machines.

My question now is how I get the different watchservices to not add the same task to the queue. Lets say machine 1 and machine 2 get the ENTRY_CREATE event at the same time and try to add it to the queue. Should I check before each

queue.offer(task)

wether that task is already present?

queue.contains(task)

Or is there a better way on the WatchService side?

Could it be bad if I have to check containsKeys(task) all the time or is this no problem? Or do you even have better ideas/approaches?

EDIT: The queue implementation I was going to use (https://docs.hazelcast.com/hazelcast/5.0/data-structures/priority-queue) provides the necessary synchronization and after reading the WatchService documentation again I also noticed, that the events do not always get processed in the same order. So my problems resolved themselves after reading those documentations again.

READ ALSO
Accessing local database from kubernetes cluster

Accessing local database from kubernetes cluster

I have a database running in local machine which i can access it using 1270

59
Handler in addValueEventListener running continuously

Handler in addValueEventListener running continuously

Im making a Uber clone app and whenever the driver receives a request he can accept it or decline it, but I want to add a feature where if the driver doesn't accept or decline the request within 8 second I want to decline it automatically

57
create SP in sybase and call it from java by passing List<XModel> to it

create SP in sybase and call it from java by passing List<XModel> to it

I have a pojo class XModeljava with fields -> f1, f2, f3, f4, f5, f6 all string and

65
Is there any way to inject Interceptor from external library in Spring?

Is there any way to inject Interceptor from external library in Spring?

I'm developing a jar library and trying to inject an interceptor from external jar library to Application

52