perf(java): prewarm configuration navigation and share results between hover and click (#947)

Configuration navigation from YAML/properties still waited for a cold index the first
time and repeated the lookup between the ⌘-hover probe and the click.

- Trusted projects with a Maven/Gradle/Eclipse build file start Java shortly after
  opening. Once JDT reports ServiceReady, the app calls the new
  openidea.java.configuration.warmup command: a low-priority background job builds
  the configuration index one project at a time (open YAML/properties modules first),
  waits for workspace builds, has a 30 s budget per project, and only fills free cache
  room.
- Index builds take a cancellable per-project lock instead of one global lock, so a
  background build never delays another project, and a warmed snapshot is dropped
  if the Java model changed meanwhile.
- In YAML/properties the editor underlines the whole key (hyphens and dots
  included), hover and Go to Definition share one bounded request per key, and the
  declaration chooser names the whole key.

Bundle java-usages 1.6.0 is rebuilt reproducibly with build.py against JDT LS
1.63.0-202610191322 (SHA-256 33cfa1bb…). The real-JDT harness checks that the first
lookup reuses the warmed index.

Co-authored-by: qjlqyf <109515705+qjlqyf@users.noreply.github.com>

java/java-usages/CONFIGURATION.md

@@ -52,12 +52,27 @@ modules, skips nested independent modules and build output, and honors ignore
 files for disk discovery. It does not guess a global association from matching
 text in unrelated projects.
 
-Indexing is lazy. The first explicit navigation/search builds a compact snapshot;
+Trusted projects with a Maven/Gradle/Eclipse build file at the root start Java a
+moment after opening, even without a Java editor open. Once JDT reports
+ServiceReady and compiler defaults are applied, the app asks the extension to warm
+the configuration index (`openidea.java.configuration.warmup`). A low-priority
+system job waits for workspace builds, then builds the same snapshot navigation
+uses, one Java project at a time: modules with open YAML/properties files first,
+then modules with open Java editors. At most 64 modules are queued, each attempt
+has a cooperative 30-second budget, repeated readiness does not repeat finished
+work, and warming only fills free room within the retention limits below (it never
+evicts an index a lookup built). A snapshot outdated while it was being built is
+dropped. Untrusted projects do not start Java; anything warming skips keeps
+on-demand indexing.
+
+Otherwise indexing is lazy. The first explicit navigation/search builds a compact snapshot;
 ordinary typing, code-lens probes, bulk scoped reference analysis, and Java Go to Definition / Ctrl+hover outside a `@Value` or property-lookup argument (checked with the bundled tree-sitter grammar) do not start indexing. At most eight project snapshots are
 retained, with a shared 100,000-association retention cap. Records contain property names, element handles and source locations;
 ASTs and bindings are discarded after indexing. A lexical prefilter avoids AST
 construction for ordinary Java units, but every accepted association is resolved
-semantically. Warm requests reuse the snapshot. Java model changes (sources on
+semantically. Builds hold a per-project lock that a waiting lookup can cancel, so a
+background build of one project never delays another, and a lookup in the same
+project reuses the result. Warm requests reuse the snapshot. Java model changes (sources on
 disk, classpath, archives) invalidate it conservatively, including dependent-type
 changes; a fingerprint of working copies with unsaved changes covers edits before
 they are saved. Opening or closing an editor, reconciling, building, and non-Java
@@ -65,6 +80,14 @@ resource changes (configuration files are parsed by the client; metadata files a
 checked by timestamp) keep it. This is **snapshot invalidation**, not a fully
 incremental Java semantic index.
 
+In YAML/properties, ⌘/Ctrl-hover underlines the whole key, hyphens and dots
+included, and doesn't open the documentation popup. Hover and Go to Definition share
+one request per key: a click after the hover reuses its answer. The editor keeps up
+to 128 answers / 4,096 locations, drops them on edits, file and project changes,
+workspace edits and language-server restarts, and expires them after 30 seconds as a
+backstop. Empty answers and errors are not kept, since the import may still be in
+progress.
+
 Metadata/JAR timestamps are checked before reuse. The client caches parsed disk
 configuration files by modification time and size (at most 100,000 cached keys), overlays open buffers, and
 performs bounded discovery/parsing on a blocking worker. It does not continuously
@@ -74,7 +97,7 @@ Bounds: 20,000 Java units, 100,000 associations per snapshot, 2 MiB per candidat
 metadata/configuration file (Java source: 2M UTF-16 units), 1,024-character property names, 2,000 configuration files/results, 50,000 filesystem
 entries per root, and depth limits on recursive bindings/YAML. Java requests and
 references are cancellable. File/association scan limits and unreadable files carry the
-existing incomplete-result indicator; unsupported syntax keys are omitted. First navigation can take seconds on a
+existing incomplete-result indicator; unsupported syntax keys are omitted. Navigation before warming finishes can take seconds on a
 cold project; classpath import and source download remain owned by JDT LS.
 
 ## Deliberate limits
@@ -105,7 +128,8 @@ cargo test -p ide-language -p ide-lsp
 The isolated real-server harness covers custom bindings, inheritance, nesting,
 collections/maps, records, constructor aliases, factory methods, consumers,
 constant keys, negative identity matches, dependency metadata/source locations,
-reverse queries, and unsaved-change/close invalidation. Optional stress fixtures
+reverse queries, warming (queued once, reused by the first lookup, cancellable), and
+unsaved-change/close invalidation. Optional stress fixtures
 add 3,000 ordinary units and 100 binding classes; output includes cold/warm/edit
 latencies for the JDT command only, excluding server import, UI and configuration-file discovery. The optional actual Spring Boot JAR checks `server.port`,
 `spring.datasource.url` and `spring.jackson.date-format` through dependency

java/java-usages/MANIFEST.MF

@@ -2,7 +2,7 @@ Manifest-Version: 1.0
 Bundle-ManifestVersion: 2
 Bundle-Name: OpenIDEA Java usages
 Bundle-SymbolicName: ai.openidea.usages;singleton:=true
-Bundle-Version: 1.5.0
+Bundle-Version: 1.6.0
 Bundle-RequiredExecutionEnvironment: JavaSE-21
 Require-Bundle: org.eclipse.jdt.ls.core,org.eclipse.jdt.core,
  org.eclipse.jdt.core.manipulation,org.eclipse.core.runtime,

java/java-usages/README.md

@@ -57,7 +57,7 @@ is not a claim of full IntelliJ feature parity.
 ## Rebuild and delivery
 
 The checked-in jar is embedded by ide-sdk `include_bytes!`, installed as
-`java-usages-1.5.0.jar`, and added alongside existing initialization bundles.
+`java-usages-1.6.0.jar`, and added alongside existing initialization bundles.
 Users need no compiler, Maven or Python. Manifest/install version must change
 when replacing the bundle because OSGi caches bundles.
 
@@ -83,6 +83,10 @@ Version 1.5.0 answers `{locations, complete}`: `complete` when the type is decla
 dependencies' implementations after the first ones. The search also covers the project's Kotlin
 classes attached by the Kotlin bridge; its reproducible SHA-256 is
 `7588748c2797f9da2beb209714a75c29b4e31b155f095329bfb7e399d807c20a`.
+Version 1.6.0 adds `openidea.java.configuration.warmup`, which builds configuration indexes in a
+low-priority background job after import, and builds them under cancellable per-project locks
+instead of one global lock; its reproducible SHA-256 is
+`33cfa1bb8489247ffeb6fc8249966c8a2955b7b2858c8169a70486eb70ade696`.
 
 ```sh
 export PATH=/path/to/jdk25/bin:$PATH

java/java-usages/plugin.xml

@@ -6,6 +6,7 @@
     </delegateCommandHandler>
     <delegateCommandHandler class="ai.openidea.configuration.ConfigurationCommand">
       <command id="openidea.java.configuration"/>
+      <command id="openidea.java.configuration.warmup"/>
     </delegateCommandHandler>
     <delegateCommandHandler class="ai.openidea.implementations.ImplementationsCommand">
       <command id="openidea.java.implementations"/>

java/java-usages/src/ai/openidea/configuration/ConfigurationCommand.java

@@ -4,7 +4,10 @@ import java.net.URI;
 import java.nio.file.Files;
 import java.nio.file.Path;
 import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicLong;
+import java.util.concurrent.locks.ReentrantLock;
 import java.util.zip.ZipFile;
 import com.google.gson.*;
 import org.eclipse.core.resources.ResourcesPlugin;
@@ -29,6 +32,8 @@ public final class ConfigurationCommand implements IDelegateCommandHandler {
     private static final int MAX_UNITS = 20_000, MAX_ENTRIES = 100_000, MAX_BYTES = 2 * 1024 * 1024;
     private static final AtomicLong REVISION = new AtomicLong(), GENERATION = new AtomicLong();
     private static final Map<String, Index> CACHE = new LinkedHashMap<>();
+    /** One build per project at a time; waiting lookups stay cancellable. */
+    private static final Map<String, ReentrantLock> LOCKS = new ConcurrentHashMap<>();
     /** Unsaved edits are covered by the working-copy digest, so reconciles never
      * invalidate. Opening/closing a working copy (F_PRIMARY_WORKING_COPY) and
      * non-Java resources, reported as F_CONTENT on a project, root or package
@@ -56,6 +61,7 @@ public final class ConfigurationCommand implements IDelegateCommandHandler {
 
     @Override
     public Object executeCommand(String command, List<Object> args, IProgressMonitor monitor) throws Exception {
+        if (command.equals("openidea.java.configuration.warmup")) return ConfigurationWarmup.request(args);
         String uri = (String) args.get(0);
         String key = args.get(1) instanceof String s ? s : null;
         int line = ((Number) args.get(2)).intValue(), column = ((Number) args.get(3)).intValue();
@@ -138,7 +144,7 @@ public final class ConfigurationCommand implements IDelegateCommandHandler {
                 (line < r.getEnd().getLine() || line == r.getEnd().getLine() && col < r.getEnd().getCharacter());
     }
     private static void check(IProgressMonitor monitor) { if (monitor.isCanceled()) throw new OperationCanceledException(); }
-    private static IJavaProject project(String uri) {
+    static IJavaProject project(String uri) {
         Path path;
         try { path = Path.of(URI.create(uri)).toFile().getCanonicalFile().toPath(); } catch (Exception e) { return null; }
         IJavaProject best = null;
@@ -163,7 +169,33 @@ public final class ConfigurationCommand implements IDelegateCommandHandler {
         return result;
     }
 
-    private static synchronized Index index(IJavaProject project, IProgressMonitor monitor) throws Exception {
+    /** Builds (or reuses) the project's index in the background; its generation. */
+    static long warm(IJavaProject project, IProgressMonitor monitor) throws Exception {
+        return index(project, monitor, true).generation;
+    }
+
+    /** Warming fills free cache room only; it never evicts an index a lookup built. */
+    static boolean hasWarmupCapacity(IJavaProject project) {
+        synchronized (CACHE) {
+            return CACHE.containsKey(project.getHandleIdentifier()) || (CACHE.size() < 8 &&
+                    CACHE.values().stream().mapToInt(value -> value.entries.size()).sum() < MAX_ENTRIES);
+        }
+    }
+
+    private static Index index(IJavaProject project, IProgressMonitor monitor) throws Exception {
+        return index(project, monitor, false);
+    }
+
+    private static Index index(IJavaProject project, IProgressMonitor monitor, boolean background) throws Exception {
+        ReentrantLock lock = LOCKS.computeIfAbsent(project.getHandleIdentifier(), ignored -> new ReentrantLock());
+        while (!lock.tryLock(50, TimeUnit.MILLISECONDS)) check(monitor);
+        try {
+            check(monitor);
+            return build(project, monitor, background);
+        } finally { lock.unlock(); }
+    }
+
+    private static Index build(IJavaProject project, IProgressMonitor monitor, boolean background) throws Exception {
         long revision = REVISION.get();
         String id = project.getHandleIdentifier();
         Map<String, ICompilationUnit> copies = new TreeMap<>();
@@ -181,7 +213,8 @@ public final class ConfigurationCommand implements IDelegateCommandHandler {
             if (source != null) digest.update(source.getBytes(java.nio.charset.StandardCharsets.UTF_8));
         }
         String workingCopies = HexFormat.of().formatHex(digest.digest());
-        Index cached = CACHE.get(id);
+        Index cached;
+        synchronized (CACHE) { cached = CACHE.get(id); }
         if (cached != null && cached.revision == revision && cached.workingCopies.equals(workingCopies)) {
             boolean valid = true;
             for (var entry : cached.files.entrySet()) {
@@ -227,11 +260,19 @@ public final class ConfigurationCommand implements IDelegateCommandHandler {
         roots.add(project.getProject().getLocationURI().toString());
         builder.metadataDirectory(project.getProject().getLocation().toFile().toPath().resolve("src/main/resources"), project);
         Index result = new Index(revision, workingCopies, List.copyOf(builder.entries), List.copyOf(roots), Map.copyOf(builder.files), builder.incomplete, builder.parsed, GENERATION.incrementAndGet());
-        CACHE.remove(id);
-        while (!CACHE.isEmpty() && (CACHE.size() >= 8 ||
-                CACHE.values().stream().mapToInt(value -> value.entries.size()).sum() + result.entries.size() > MAX_ENTRIES))
-            CACHE.remove(CACHE.keySet().iterator().next());
-        CACHE.put(id, result);
+        check(monitor);
+        // A warmed index that the model has already outdated is dropped, not published.
+        if (background && revision != REVISION.get()) throw new OperationCanceledException();
+        synchronized (CACHE) {
+            if (background && ((!CACHE.containsKey(id) && CACHE.size() >= 8) ||
+                    CACHE.entrySet().stream().filter(e -> !e.getKey().equals(id)).mapToInt(e -> e.getValue().entries.size()).sum()
+                            + result.entries.size() > MAX_ENTRIES)) throw new OperationCanceledException();
+            CACHE.remove(id);
+            while (!CACHE.isEmpty() && (CACHE.size() >= 8 ||
+                    CACHE.values().stream().mapToInt(value -> value.entries.size()).sum() + result.entries.size() > MAX_ENTRIES))
+                CACHE.remove(CACHE.keySet().iterator().next());
+            CACHE.put(id, result);
+        }
         return result;
     }
 

java/java-usages/src/ai/openidea/configuration/ConfigurationWarmup.java

@@ -0,0 +1,111 @@
+package ai.openidea.configuration;
+
+import java.util.*;
+import org.eclipse.core.resources.ResourcesPlugin;
+import org.eclipse.core.runtime.*;
+import org.eclipse.core.runtime.jobs.Job;
+import org.eclipse.jdt.core.*;
+
+/** One bounded, cooperative queue per language-server process. No polling timer,
+ * retained ASTs, or project builds; an ordinary lookup remains the fallback. */
+final class ConfigurationWarmup {
+    private static final Map<String, IJavaProject> pending = new LinkedHashMap<>();
+    private static final Set<String> attempted = new HashSet<>();
+    private static final Map<String, Long> completed = new LinkedHashMap<>();
+    private static boolean scheduled, stopped;
+    private static String active;
+    private static final Job job = new Job("Preparing configuration navigation") {
+        @Override protected IStatus run(IProgressMonitor monitor) {
+            // Compiler defaults/import can schedule a build after ServiceReady.
+            // Yield to it instead of blocking a worker on the build job family.
+            if (building()) {
+                synchronized (ConfigurationWarmup.class) {
+                    if (!stopped) schedule(2000);
+                    else scheduled = false;
+                }
+                return Status.OK_STATUS;
+            }
+            IJavaProject project;
+            synchronized (ConfigurationWarmup.class) {
+                if (stopped || pending.isEmpty()) { scheduled = false; return Status.OK_STATUS; }
+                String id = pending.keySet().iterator().next();
+                project = pending.remove(id);
+                active = id;
+            }
+            long deadline = System.nanoTime() + 30_000_000_000L;
+            IProgressMonitor bounded = new ProgressMonitorWrapper(monitor) {
+                @Override public boolean isCanceled() {
+                    return super.isCanceled() || System.nanoTime() >= deadline ||
+                            !project.exists() || !project.getProject().isOpen();
+                }
+            };
+            try {
+                if (ConfigurationCommand.hasWarmupCapacity(project)) {
+                    long build = ConfigurationCommand.warm(project, bounded);
+                    synchronized (ConfigurationWarmup.class) {
+                        if (completed.size() < 8) completed.put(project.getHandleIdentifier(), build);
+                    }
+                }
+            } catch (OperationCanceledException e) {
+                // No partial index is published. A user lookup can retry later.
+            } catch (Exception e) {
+                org.eclipse.jdt.ls.core.internal.JavaLanguageServerPlugin.logException("Configuration warmup skipped", e);
+            } finally {
+                synchronized (ConfigurationWarmup.class) {
+                    active = null;
+                    if (!stopped && !pending.isEmpty()) schedule(200);
+                    else scheduled = false;
+                }
+            }
+            return Status.OK_STATUS;
+        }
+    };
+    static { job.setSystem(true); job.setPriority(Job.DECORATE); }
+
+    private static boolean building() {
+        for (Object family : new Object[] { ResourcesPlugin.FAMILY_AUTO_BUILD, ResourcesPlugin.FAMILY_MANUAL_BUILD })
+            for (Job other : Job.getJobManager().find(family))
+                if (other.getState() != Job.NONE) return true;
+        return false;
+    }
+
+    static synchronized Object request(List<Object> args) throws JavaModelException {
+        String action = args.isEmpty() ? "start" : String.valueOf(args.get(0));
+        if (action.equals("cancel")) {
+            stopped = true;
+            pending.clear();
+            job.cancel();
+            if (active == null) scheduled = false;
+        } else if (action.equals("start") && !stopped) {
+            // Open Java buffers identify the user's active modules. Keep those
+            // ahead of the rest when the workspace exceeds the cache budget.
+            Set<String> open = new HashSet<>();
+            for (ICompilationUnit unit : JavaCore.getWorkingCopies(null)) open.add(unit.getJavaProject().getHandleIdentifier());
+            for (ICompilationUnit unit : JavaCore.getWorkingCopies(
+                    org.eclipse.jdt.ls.core.internal.JavaLanguageServerPlugin.getInstance().getWorkingCopyOwner()))
+                open.add(unit.getJavaProject().getHandleIdentifier());
+            Set<String> preferred = new HashSet<>();
+            if (args.size() > 1 && args.get(1) instanceof List<?> uris) {
+                for (Object uri : uris) if (uri instanceof String text) {
+                    IJavaProject project = ConfigurationCommand.project(text);
+                    if (project != null) preferred.add(project.getHandleIdentifier());
+                }
+            }
+            List<IJavaProject> projects = new ArrayList<>(Arrays.asList(
+                    JavaCore.create(ResourcesPlugin.getWorkspace().getRoot()).getJavaProjects()));
+            projects.sort(Comparator.comparing((IJavaProject p) -> !preferred.contains(p.getHandleIdentifier()))
+                    .thenComparing(p -> !open.contains(p.getHandleIdentifier()))
+                    .thenComparing(IJavaProject::getElementName));
+            for (IJavaProject project : projects) {
+                String id = project.getHandleIdentifier();
+                if (project.getProject().isOpen() && !attempted.contains(id) && pending.size() < 64) {
+                    attempted.add(id);
+                    pending.put(id, project);
+                }
+            }
+            if (!scheduled && !pending.isEmpty()) { scheduled = true; job.schedule(2000); }
+        }
+        return Map.of("scheduled", scheduled, "pending", pending.size(), "active", active == null ? "" : active,
+                "completed", new LinkedHashMap<>(completed));
+    }
+}

java/java-usages/tests/configuration.py

@@ -159,11 +159,28 @@ try:
     send({'jsonrpc': '2.0', 'method': 'textDocument/didOpen', 'params': {'textDocument': {'uri': file.as_uri(), 'languageId': 'java', 'version': 1, 'text': source}}})
     request('java/buildWorkspace', True)
     assert 'openidea.java.configuration' in init['capabilities']['executeCommandProvider']['commands'], init
+    def warmup(action='start'):
+        return request('workspace/executeCommand', {'command': 'openidea.java.configuration.warmup', 'arguments': [action, [(project / 'application.yml').as_uri()]]})
+    # Queuing is cheap and idempotent; the index is built by a background job.
+    queued_at = time.perf_counter()
+    queued = warmup()
+    assert queued['scheduled'] and queued['pending'] > 0 and not queued['completed'], queued
+    assert warmup()['pending'] == queued['pending'], 'a repeated start queued the project twice'
+    deadline = time.monotonic() + 90
+    while time.monotonic() < deadline:
+        warmed = warmup('status')
+        if not warmed['scheduled']: break
+        time.sleep(.1)
+    assert not warmed['scheduled'] and warmed['completed'], warmed
+    assert warmup()['pending'] == 0, 'a repeated start queued a warmed project again'
+    prewarm_seconds = time.perf_counter() - queued_at
     results = {}
     for key in ['app', 'vendor', 'app.time-out', 'app.enabled', 'app.nested.url', 'app.clients[0].url', 'app.routes.east.url', 'app.tags[0]', 'app.labels.region', 'app.labels[region.name]', 'constructor.remote-name', 'record.name', 'bean.url', 'app.backup', 'app.required', 'vendor.port', 'extra.url']:
         start = time.perf_counter()
         result = query(key)
         results[key] = {'seconds': time.perf_counter() - start, 'response': result}
+        if key == 'app':
+            assert result['generation'] in warmed['completed'].values(), ('the first lookup rebuilt the warmed index', result, warmed)
         assert result['locations'], (key, result)
     attached = request('java/classFileContents', {'uri': results['vendor.port']['response']['locations'][0]['uri']})
     assert attached and 'LibrarySettings' in attached and 'port' in attached, attached
@@ -238,7 +255,9 @@ try:
     send({'jsonrpc': '2.0', 'method': 'textDocument/didClose', 'params': {'textDocument': {'uri': file.as_uri()}}})
     restored = query('app.nested.url')
     assert restored['locations'], restored
-    evidence = {'stress': os.environ.get('OPENIDEA_CONFIGURATION_STRESS') == '1', 'editRebuildMs': edit_ms, 'cases': results, 'warmMedianMs': sorted(warm)[len(warm)//2] * 1000, 'warmMaxMs': max(warm) * 1000, 'generations': [generation, after_save, after_close]}
+    cancelled = warmup('cancel')
+    assert not cancelled['scheduled'] and cancelled['pending'] == 0, cancelled
+    evidence = {'stress': os.environ.get('OPENIDEA_CONFIGURATION_STRESS') == '1', 'prewarmSeconds': prewarm_seconds, 'editRebuildMs': edit_ms, 'cases': results, 'warmMedianMs': sorted(warm)[len(warm)//2] * 1000, 'warmMaxMs': max(warm) * 1000, 'generations': [generation, after_save, after_close]}
     (work / 'evidence.json').write_text(json.dumps(evidence, indent=2))
     print(json.dumps(evidence, indent=2))
 finally: