Upgrade directory-watcher library (#8201)

The change upgrades `directory-watcher` library, hoping that it will fix the problem reported in #7695 (there has been a number of bug fixes in MacOS listener since then).

Once upgraded, tests in `WatcherAdapterSpec` because the logic that attempted to ensure the proper initialization order in the test using semaphore was wrong. Now starting the watcher using `watchAsync` which only returns the future when the watcher successfully registers for paths. Ideally authors of the library would make the registration bit public
(3218d68a84/core/src/main/java/io/methvin/watcher/DirectoryWatcher.java (L229C7-L229C20)) but it is the best we can do so far.

Had to adapt to the new API in PathWatcher as well, ensuring the right order of initialization.

Should fix #7695.
This commit is contained in:
Hubert Plociniczak 2023-11-02 12:24:26 +01:00 committed by GitHub
parent a9118ee0c3
commit 2db4f4c5d9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
24 changed files with 172 additions and 74 deletions

View File

@ -460,7 +460,7 @@ val zio = Seq(
val bcpkixJdk15Version = "1.70"
val bumpVersion = "0.1.3"
val declineVersion = "2.4.1"
val directoryWatcherVersion = "0.9.10"
val directoryWatcherVersion = "0.18.0"
val flatbuffersVersion = "1.12.0"
val guavaVersion = "32.0.0-jre"
val jlineVersion = "3.23.0"

View File

@ -253,7 +253,7 @@ Copyright notices related to this dependency can be found in the directory `io.c
'directory-watcher', licensed under the Apache-2.0, is distributed with the engine.
The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `io.methvin.directory-watcher-0.9.10`.
Copyright notices related to this dependency can be found in the directory `io.methvin.directory-watcher-0.18.0`.
'spray-json_2.13', licensed under the Apache 2, is distributed with the engine.
@ -261,9 +261,9 @@ The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `io.spray.spray-json_2.13-1.3.6`.
'jna', licensed under the Apache License v2.0, is distributed with the engine.
'jna', licensed under the Apache-2.0, is distributed with the engine.
The license file can be found at `licenses/APACHE2.0`.
Copyright notices related to this dependency can be found in the directory `net.java.dev.jna.jna-5.5.0`.
Copyright notices related to this dependency can be found in the directory `net.java.dev.jna.jna-5.12.1`.
'bump_2.13', licensed under the MIT, is distributed with the engine.

View File

@ -0,0 +1,5 @@
Apache Commons Codec
Copyright 2002-2023 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (https://www.apache.org/).

View File

@ -1 +1,5 @@
Some portions of this file Copyright (c) 2004-2006 Intel Corporation
this work for additional information regarding copyright ownership.

View File

@ -0,0 +1,25 @@
Copyright (c) 2007 Timothy Wall
Copyright (c) 2007 Timothy Wall, All Rights Reserved
Copyright (c) 2007 Wayne Meissner, All Rights Reserved
Copyright (c) 2007-2008 Timothy Wall, All Rights Reserved
Copyright (c) 2007-2012 Timothy Wall, All Rights Reserved
Copyright (c) 2007-2013 Timothy Wall, All Rights Reserved
Copyright (c) 2007-2015 Timothy Wall, All Rights Reserved
Copyright (c) 2009 Timothy Wall, All Rights Reserved
Copyright (c) 2011 Timothy Wall, All Rights Reserved
Copyright (c) 2012 Timothy Wall, All Rights Reserved
Copyright (c) 2017 Matthias Bläsing, All Rights Reserved
Copyright (c) 2018 Matthias Bläsing
Copyright (c) 2019 Matthias Bläsing, All Rights Reserved

View File

@ -1,5 +0,0 @@
Copyright (c) 2007 Timothy Wall, All Rights Reserved
Copyright (c) 2007 Wayne Meissner, All Rights Reserved
Copyright (c) 2017 Matthias Bläsing, All Rights Reserved

View File

@ -88,6 +88,8 @@ Copyright (C) 2008, 2022 Marek Zawirski <marek.zawirski@gmail.com> and others
Copyright (C) 2008, 2022 Shawn O. Pearce <spearce@spearce.org> and others
Copyright (C) 2008, 2023 Google Inc. and others
Copyright (C) 2008, Charles O'Farrell <charleso@charleso.org>
Copyright (C) 2008, Florian Köberle <florianskarten@web.de>
@ -338,8 +340,6 @@ Copyright (C) 2011, 2019 GitHub Inc. and others
Copyright (C) 2011, 2019 Google Inc. and others
Copyright (C) 2011, 2020 Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2011, 2020, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2011, 2021 IBM Corporation and others
@ -350,6 +350,8 @@ Copyright (C) 2011, 2022 Christoph Brill <egore911@egore911.de> and others
Copyright (C) 2011, 2022 Google Inc. and others
Copyright (C) 2011, 2023 Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2011, Chris Aniszczyk <caniszczyk@gmail.com>
Copyright (C) 2011, Chris Aniszczyk <caniszczyk@gmail.com> and others
@ -398,10 +400,10 @@ Copyright (C) 2012 Christian Halstrick and others
Copyright (C) 2012 Google Inc. and others
Copyright (C) 2012, 2021 GitHub Inc. and others
Copyright (C) 2012, 2022, Robin Rosenberg and others
Copyright (C) 2012, 2023 GitHub Inc. and others
Copyright (C) 2012, Christian Halstrick <christian.halstrick@sap.com> and others
Copyright (C) 2012, Daniel Megert <daniel_megert@ch.ibm.com>
@ -638,6 +640,10 @@ Copyright (C) 2021, Google Inc. and others
Copyright (C) 2021, Google LLC. and others
Copyright (C) 2021, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2021, Tencent.
Copyright (C) 2021, Thomas Wolf <thomas.wolf@paranor.ch> and others
Copyright (C) 2022 Thomas Wolf <thomas.wolf@paranor.ch> and others
@ -646,7 +652,9 @@ Copyright (C) 2022, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2022, Fabio Ponciroli <ponch78@gmail.com> and others
Copyright (C) 2022, Google Inc. and others
Copyright (C) 2022, Google LLC and others
Copyright (C) 2022, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2022, Simeon Andreev and others
@ -654,6 +662,20 @@ Copyright (C) 2022, Tencent.
Copyright (C) 2022, Workday Inc.
Copyright (C) 2023, Google Inc. and others
Copyright (C) 2023, Google LLC
Copyright (C) 2023, Google LLC and others
Copyright (C) 2023, Google LLC.
Copyright (C) 2023, SAP SE and others
Copyright (C) 2023, Tencent.
Copyright (C) 2023, Thomas Wolf <twolf@apache.org> and others
Copyright (c) 2014 Konrad Kügler and others
Copyright (c) 2019 Matthias Sohn <matthias.sohn@sap.com>
@ -672,6 +694,8 @@ Copyright (c) 2020, Google LLC and others
Copyright (c) 2021 Qualcomm Innovation Center, Inc.
Copyright (c) 2023 Qualcomm Innovation Center, Inc.
Copyright 2017 Marc Stevens <marc@marc-stevens.nl>, Dan Shumow <danshu@microsoft.com>
IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,

View File

@ -14,7 +14,8 @@ trait Exec[-F[_, _]] {
/** Execute Zio effect.
*
* @param op effect to execute
* @param op effect to execute
* @param trace object prevents the library from messing up the user's execution trace
* @return a future containing either a failure or a result
*/
def exec[E, A](op: F[E, A])(implicit trace: Trace): Future[Either[E, A]]
@ -22,8 +23,10 @@ trait Exec[-F[_, _]] {
/** Execute Zio effect with timeout.
*
* @param timeout execution timeout
* @param op effect to execute
* @return a future
* @param op effect to execute
* @param trace object prevents the library from messing up the user's execution trace
* @return a future. On timeout future is failed with `TimeoutException`.
* Otherwise future contains either a failure or a result.
*/
def execTimed[E, A](
timeout: FiniteDuration,
@ -32,9 +35,19 @@ trait Exec[-F[_, _]] {
/** Execute long running task in background.
*
* @param op effect to execute
* @param op effect to execute
* @param trace object prevents the library from messing up the user's execution trace
*/
def exec_[E <: Throwable, A](op: F[E, A])(implicit trace: Trace): Unit
/** Execute long running task in background.
*
* @param op effect to execute with the explicit executor
* @param trace object prevents the library from messing up the user's execution trace
*/
def exec__[E <: Throwable, A](op: Executor => F[E, A])(implicit
trace: Trace
): Unit
}
/** Executor of Zio effects.
@ -43,12 +56,7 @@ trait Exec[-F[_, _]] {
*/
case class ZioExec(runtime: effect.Runtime) extends Exec[ZioExec.IO] {
/** Execute Zio effect.
*
* @param op effect to execute
* @param trace object prevents the library from messing up the user's execution trace
* @return a future containing either a failure or a result
*/
/** @inheritdoc */
override def exec[E, A](
op: ZIO[ZAny, E, A]
)(implicit trace: Trace): Future[Either[E, A]] =
@ -63,14 +71,7 @@ case class ZioExec(runtime: effect.Runtime) extends Exec[ZioExec.IO] {
promise.future
}
/** Execute Zio effect with timeout.
*
* @param timeout execution timeout
* @param op effect to execute
* @param trace object prevents the library from messing up the user's execution trace
* @return a future. On timeout future is failed with `TimeoutException`.
* Otherwise future contains either a failure or a result.
*/
/** @inheritdoc */
override def execTimed[E, A](
timeout: FiniteDuration,
op: ZIO[ZAny, E, A]
@ -90,17 +91,23 @@ case class ZioExec(runtime: effect.Runtime) extends Exec[ZioExec.IO] {
promise.future
}
/** Execute long running task in background.
*
* @param op effect to execute
* @param trace object prevents the library from messing up the user's execution trace
*/
/** @inheritdoc */
override def exec_[E <: Throwable, A](
op: ZIO[ZAny, E, A]
)(implicit trace: Trace): Unit =
zio.Unsafe.unsafe { implicit unsafe =>
runtime.instance.unsafe.fork(ZIO.blocking(op))
}
/** @inheritdoc */
override def exec__[E <: Throwable, A](
op: Executor => ZioExec.IO[E, A]
)(implicit trace: Trace): Unit =
zio.Unsafe.unsafe { implicit unsafe =>
runtime.instance.unsafe.fork(
ZIO.blockingExecutor.flatMap(ec => op(ec).onExecutor(ec))
)
}
}
object ZioExec {

View File

@ -23,11 +23,11 @@ import org.enso.languageserver.filemanager.PathWatcher.{
ForwardResponse
}
import org.enso.languageserver.util.UnhandledLogging
import zio._
import zio.ZIO
import java.io.File
import scala.concurrent.Await
import scala.util.{Failure, Success}
import scala.util.{Failure, Success, Try}
/** Starts [[Watcher]], handles errors, converts and sends
* events to the client.
@ -81,7 +81,7 @@ final class PathWatcher(
_ <- ZIO.fromEither(startWatcher(watcher))
} yield ()
exec
val executedResult = exec
.exec(result)
.map {
case Right(()) => CapabilityAcquired
@ -89,15 +89,29 @@ final class PathWatcher(
}
.pipeTo(sender())
pathToWatchResult.onComplete {
case Success(Right(root)) =>
logger.info("Initialized [{}] for [{}].", watcherFactory.getClass, path)
context.become(initializedStage(root, path, clients))
case Success(Left(err)) =>
logger.error("Failed to resolve the path [{}]. {}", path, err)
context.stop(self)
Try(Await.ready(executedResult, config.timeout)) match {
case Success(_) =>
pathToWatchResult.onComplete {
case Success(Right(root)) =>
logger.info(
"Initialized [{}] for [{}].",
watcherFactory.getClass,
path
)
context.become(initializedStage(root, path, clients))
case Success(Left(err)) =>
logger.error("Failed to resolve the path [{}]. {}", path, err)
context.stop(self)
case Failure(err) =>
logger.error("Failed to resolve the path [{}]", path, err)
context.stop(self)
}
case Failure(err) =>
logger.error("Failed to resolve the path [{}]", path, err)
logger.error(
"Failed to initialize path watcher for path [{}]. {}",
path,
err
)
context.stop(self)
}
}
@ -182,7 +196,7 @@ final class PathWatcher(
Either
.catchNonFatal {
fileWatcher = Some(watcher)
exec.exec_(ZIO.attempt(watcher.start()))
exec.exec__(ec => ZIO.attempt(watcher.start(ec.asJava)))
}
.leftMap(errorHandler)

View File

@ -1,10 +1,12 @@
package org.enso.filewatcher
import java.util.concurrent.Executor
/** A file watcher that does nothing. */
class NoopWatcher extends Watcher {
/** @inheritdoc */
override def start(): Unit = ()
override def start(ec: Executor): Unit = ()
/** @inheritdoc */
override def stop(): Unit = ()

View File

@ -1,11 +1,15 @@
package org.enso.filewatcher
import java.nio.file.Path
import java.util.concurrent.Executor
trait Watcher {
/** Start the file watcher. */
def start(): Unit
/** Start the file watcher.
*
* @param executor executor under which this watcher should be run in the background
*/
def start(executor: Executor): Unit
/** Stop the file watcher. */
def stop(): Unit

View File

@ -1,7 +1,12 @@
package org.enso.filewatcher
import io.methvin.watcher._
import io.methvin.watcher.{
DirectoryChangeEvent,
DirectoryChangeListener,
DirectoryWatcher
}
import java.util.concurrent.Executor
import java.nio.file.Path
/** Watches the root with subdirectories and executes callback on file event.
@ -24,8 +29,8 @@ final class WatcherAdapter(
.build()
/** @inheritdoc */
override def start(): Unit = {
watcher.watch()
override def start(executor: Executor): Unit = {
watcher.watchAsync(executor)
}
/** @inheritdoc */

View File

@ -1,9 +1,9 @@
package org.enso.filewatcher
import java.nio.file.{Files, Path, Paths}
import java.util.concurrent.{Executors, LinkedBlockingQueue, Semaphore}
import org.apache.commons.io.FileUtils
import java.nio.file.{Files, Path, Paths}
import java.util.concurrent.{Executors, LinkedBlockingQueue}
import org.enso.testkit.RetrySpec
import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers
@ -66,19 +66,14 @@ class WatcherAdapterSpec extends AnyFlatSpec with Matchers with RetrySpec {
def withWatcher(
test: (Path, LinkedBlockingQueue[Watcher.WatcherEvent]) => Any
): Any = {
val lock = new Semaphore(0)
val executor = Executors.newSingleThreadExecutor()
val tmp = Files.createTempDirectory(null).toRealPath()
val queue = new LinkedBlockingQueue[Watcher.WatcherEvent]()
val watcher = new WatcherAdapterFactory().build(tmp, queue.put, println(_))
executor.submit[Any] { () =>
lock.release()
watcher.start()
}
watcher.start(executor)
try {
lock.tryAcquire(Timeout.length, Timeout.unit)
test(tmp, queue)
} finally {
watcher.stop()

View File

@ -0,0 +1,2 @@
<p>This is public domain code with no copyrights. From home page of
This is public domain code with no copyrights. From home page of

View File

@ -0,0 +1,2 @@
Some portions of this file Copyright (c) 2004-2006 Intel Corporation
this work for additional information regarding copyright ownership.

View File

@ -0,0 +1 @@
META-INF/LICENSE.txt

View File

@ -0,0 +1 @@
META-INF/NOTICE.txt

View File

@ -0,0 +1 @@
Copyright 2007 Timothy Wall

View File

@ -1,11 +1,13 @@
Copyright (c) 2007 Timothy Wall
Copyright (c) 2007 Timothy Wall, All Rights Reserved
Copyright (c) 2007 Wayne Meissner, All Rights Reserved
Copyright (c) 2007-2008 Timothy Wall, All Rights Reserved
Copyright (c) 2007-2012 Timothy Wall, All Rights Reserved
Copyright (c) 2007-2013 Timothy Wall, All Rights Reserved
Copyright (c) 2007-2015 Timothy Wall, All Rights Reserved
Copyright (c) 2009 Timothy Wall, All Rights Reserved
Copyright (c) 2011 Timothy Wall, All Rights Reserved
Copyright (c) 2012 Timothy Wall, All Rights Reserved
Copyright (c) 2017 Matthias Bläsing, All Rights Reserved
Copyright (c) 2018 Matthias Bläsing
Copyright (c) 2019 Matthias Bläsing, All Rights Reserved
Copyright 2007 Timothy Wall
Copyright (c) 2007-2012 Timothy Wall, All Rights Reserved

View File

@ -1,3 +0,0 @@
Copyright (c) 2007 Timothy Wall, All Rights Reserved
Copyright (c) 2007 Wayne Meissner, All Rights Reserved
Copyright (c) 2017 Matthias Bläsing, All Rights Reserved

View File

@ -70,7 +70,6 @@ Copyright (C) 2008, Imran M Yousuf <imyousuf@smartitengineering.com>
Copyright (C) 2021, Google Inc. and others
Copyright (C) 2022 Thomas Wolf <thomas.wolf@paranor.ch> and others
Copyright (C) 2022, Simeon Andreev and others
Copyright (C) 2022, Google Inc. and others
Copyright (C) 2022, Fabio Ponciroli <ponch78@gmail.com> and others
Copyright (C) 2022, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2021, Thomas Wolf <thomas.wolf@paranor.ch> and others
@ -202,7 +201,6 @@ Copyright (C) 2011, 2013 Google Inc., and others. and others
Copyright (C) 2011, 2013 Robin Rosenberg and others
Copyright (C) 2011, 2019 GitHub Inc. and others
Copyright (C) 2011, 2019 Google Inc. and others
Copyright (C) 2011, 2020 Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2011, 2020, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2011, 2021 IBM Corporation and others
Copyright (C) 2011, 2022 Chris Aniszczyk <caniszczyk@gmail.com> and others
@ -232,7 +230,6 @@ Copyright (C) 2011-2013, Chris Aniszczyk <caniszczyk@gmail.com> and others
Copyright (C) 2011-2013, Robin Rosenberg <robin.rosenberg@dewire.com> and others
Copyright (C) 2012 Christian Halstrick and others
Copyright (C) 2012 Google Inc. and others
Copyright (C) 2012, 2021 GitHub Inc. and others
Copyright (C) 2012, 2022, Robin Rosenberg and others
Copyright (C) 2012, Christian Halstrick <christian.halstrick@sap.com> and others
Copyright (C) 2012, Daniel Megert <daniel_megert@ch.ibm.com>
@ -339,3 +336,18 @@ other copyright owners as documented in the project's IP log.
copyright notice, this list of conditions and the following
Copyright (C) 2010, Mathias Kinzler <mathias.kinzler@sap.com> and
Copyright (C) 2010, Christian Halstrick <christian.halstrick@sap.com> and others
Copyright (C) 2021, Tencent.
Copyright (C) 2021, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2022, Matthias Sohn <matthias.sohn@sap.com> and others
Copyright (C) 2023, Google Inc. and others
Copyright (C) 2023, Google LLC
Copyright (C) 2023, Google LLC and others
Copyright (C) 2023, Google LLC.
Copyright (C) 2023, SAP SE and others
Copyright (C) 2023, Tencent.
Copyright (C) 2023, Thomas Wolf <twolf@apache.org> and others
Copyright (c) 2023 Qualcomm Innovation Center, Inc.
Copyright (C) 2022, Google LLC and others
Copyright (C) 2008, 2023 Google Inc. and others
Copyright (C) 2012, 2023 GitHub Inc. and others
Copyright (C) 2011, 2023 Matthias Sohn <matthias.sohn@sap.com> and others

View File

@ -1,3 +1,3 @@
BBA040F93183A17D3A861F0013540424DEBD31136C001A4653F7C6CEFC212A78
3933B106917A6381759200AEE4B93A368AE355CB8DFF87C1819A346C54D91E0E
111746D8663350778901C6EE9D428BCA659E04160F16055FF33CDD29294D03DB
D6B194A8D9FE9212D9077E0A83DF32895A4840E704817E4479D98B4329497B49
0