DMTN-349

Experimenting with Porting a Large C Library to Rust Using Coding Agents#

Abstract

With the rapid rise in abilities of Large Language Model Coding Agents such as Codex and Claude Code over the course of 2026, it became apparent that some software tasks which had seemed impossible to implement were now possible to contemplate. In this document I will discuss one such task involving the conversion of approximately 475,000 lines of a C library that has been in development for 30 years to Rust. The motivation for choosing Rust is the much stronger memory management model and additional type safety inherent in the language, coupled with the more modern tooling infrastructure available and the potential to support technologies such as WebAssembly.

The main outcome from this work is that your success in porting any code to a different language using agents depends entirely on the test coverage of your original source code.

Introduction#

In February 2026 I was modernizing the Python bindings of the Starlink AST library [Berry et al., 2016] to make it usable as the WCS backend for the file format and data modeling rewrite of the LSST Science Pipelines images [Bosch, 2026]. This was needed since there had been a new release of the C library that included critical functionality and Rubin needed those changes on PyPI. For historical reasons (the Python wrapping work started in 2011) the Python interface was written using native C Python APIs and had to support Python 2.6 and 3.x. I decided to try using the newly-released OpenAI Codex with GPT-5.3 and I was impressed by its ability to quickly analyze the code, remove the Python 2 legacy interfaces and modernize the C API calls for Python 3.11 and newer. Since I had long wondered whether the wrapper should be using Cython instead of native APIs, I asked Codex to make a proof of concept demonstration of a Cython wrapper and it did exactly that. For context the hand-crafted C wrapper consisted of about 13,000 lines of code, so a full conversion was not a trivial undertaking but the demonstration did hint at future possibilities. This proof of concept led to a discussion of the broader implications and I wondered if I could now port the AST C library to Rust.

The long-term support of the C library has been a concern in the project ever since Rubin adopted the library for WCS in 2016 [Parejko and Owen, 2016]. In particular the object-oriented C style and steep learning curve, associated with the library having a single developer, caused concern about the long-term support of the library[1] and some people at Rubin had been wondering whether it would be better for the next decade if there were a Rust library implementing the core AST algorithms that the project needed.

People have been wondering if large language models can aid with language ports for a while [Dehghan et al., 2025, Ibrahimzada et al., 2025] but the state of the art in this field is moving so fast that techniques and failures from 2025 using models like ChatGPT 4 are fully out of date. In Dehghan et al. [2025] they built a specialist infrastructure specifically for using LLMs to convert from C to Rust but the largest code base they tackled was of order 15,000 lines of code. More recently, May 2026, there was a demonstration of a port of the 500k LOC ngspice package from C to Rust, spice-rs. The Git repository does not include any of the porting history and so it is difficult to say how the codebase evolved but the goal of the port was similar to the goals described here: a faithful port to Rust with byte-exact conformance with the original library.

A port of this size inevitably means that it’s impossible to review every line of code and there is a danger that the entire port is effectively “vibe-coded” [Elgendy et al., 2026, Sarkar and Drosos, 2025] as more and more code flies by and you always accept each suggestion. For a small throw-away application this is an acceptable approach, but it comes with risks when porting a large library that is intended for production use.

Previous Attempts at Porting the AST Library#

In the early 2000s the Starlink Project was instructed to switch away from using C and Fortran with a bespoke message bus (ADAM) and instead switch to the more modern technologies of Java and SOAP [Bly et al., 2003]. As part of that work it was realized that the AST library was a core foundation of the Starlink software ecosystem. They started work on such a port in early 2002 but by 2002 October they had already decided that as an interim measure there should be a JNI wrapper around the existing AST C library.[2] This has many disadvantages in that it requires a different binary per supported architecture but it did unblock the use of WCS in tools such as SPLAT [Škoda et al., 2014], SoG (Son of GAIA), and Treeview.[3] By the time of the 2003 ADASS [Berry, 2004] there was no mention of a native Java port, solely JNI, and the project had been abandoned.

There is no public record of the project’s demise, but later discussions (Berry, private communication) indicated that the project involved writing Perl scripts to try to recognize the standard boilerplate in the files and extract the business logic in a way that could be more easily converted to Java. The project failed in part due to the complexities of mapping C pointer behavior to Java. There was also some concern that the legacy FITS header support would be particularly challenging to reproduce.

AST Usage in the LSST Science Pipelines#

There were two aspects to integrating the AST library into the LSST Science Pipelines [Rubin Observatory Science Pipelines Developers et al., 2026]. First a C++ wrapper was written[4] to provide an interface more similar to existing C++ semantics used in the pipelines code. This included the addition of more controlled object lifetimes and standardized property accessors.

The second aspect was to wrap the AST internals in a more specialist SkyWcs C++ class such that a pipelines user would never know how the WCS transforms were implemented unless they were explicitly manipulating the mappings. The AST plain text serialization was then used when writing the object to FITS. This approach was also adopted by the lsst-images package [Bosch, 2026] where the AST internals (this time also backed by the PyPI Python wrapper) are abstracted behind a SkyProjection class and the plain text is stored in the JSON serialized model in files.

Porting to Rust#

As the C++ standard has become more and more complex, resulting in books actively telling you all the ways you can make mistakes [Lakos et al., 2021, Meyers, 2014, Yonts, 2025], Rubin increasingly sought alternatives. Rubin Observatory has been considering the use of Rust [Klabnik et al., 2026] for some time, with RFC-1069 in early 2025 approving its usage in the LSST Science Pipelines. The first Rust code was added to the LSST Science Pipelines [Rubin Observatory Science Pipelines Developers et al., 2026] in early 2026.[5]

There are numerous reasons to consider porting the C AST library to Rust.

  1. The bespoke object-oriented implementation comes with a lot of boilerplate code and a steep learning curve. Rust includes native support for many equivalent concepts, raising the possibility of the same functionality using far less code and being more focused on the algorithms that matter.

  2. Memory safety is a core goal of Rust with a very strong ownership and borrowing model making many coding errors relating to memory management and buffer overflows impossible.

  3. Rust has a much stronger type system than C, enforced by the compiler, and has explicit approaches for indicating optional return values and error states that can be reasoned about by the compiler and are much easier to track than the AST global status.

Additionally, once a decision has been made to port to Rust there are well-supported ways for providing a backwards compatible C interface as well as the possibility of having a clean independent Python wrapper directly on the Rust. These reasons were enough to suggest that I should try to port the AST library, even though it is clearly a very challenging library to pick as a first test of using agents to do a port.

The First Prototype#

In order to investigate the feasibility of a port I decided the easiest thing to do was to try it and see what would happen. My previous experience of using coding agents was for small fixes and code cleanups as described in the introduction. Late in the afternoon of 2026-03-03 I decided to try the OpenAI Codex application (model GPT-5.3) to start working on the port. I was using a $20/month Team account for this work. At the time I did not know enough to make sure there was an AGENTS.md file to refer to repository policies and was not aware that much can be gained by explicitly planning more up front.

My first prompt was:

I am attempting to port the C library in the ast/ subdirectory to rust. A description of the plan with more instructions can be found in the file TODO.md.

Please read that file and come up with an initial plan for the port. Append to TODO.md as needed to remember decisions.

and my associated TODO.md support file was:

The aim of this work is to convert the Starlink AST C library (in the ast directory) into a rust library in the starlink-astrs directory.

The rust library should be object-oriented like the original and should follow all standard rust packaging and linting. It should also include C bindings so that it can replace the C version and to allow easy testing.

The documentation describing the C library is found in sun211.pdf.

The C code has a lot of standard boiler plate trying to emulate an object-oriented library in C without C++. Work was started on the package before C++ was mainstream. A full git history can be found in the ast/ subdirectory. The rust version can drop the boiler plate and focus on the business logic implementing the frames and mappings.

You may append to the file in the PORT.md file with notes as needed to remember previous decisions

where I had made available the AST C source code in a parallel tree. It then wrote an 80-line outline of a plan and started work. By the evening of 2026-03-05 (two days later) I had 25,000 lines of Rust library code and 12,000 lines of tests and I had run out of my weekly allocation of tokens on the cheap Codex plan. This seemed incredibly promising. The coding agent was able to build the C library and write exploratory code against it to act as an “oracle” (its term) with which to compare the Rust code.

Work resumed towards the end of March and whilst initial progress was good, adding another 10,000 lines in a day, work then became much more difficult. As the porting evolved I decided to focus on a core functionality of being able to read a Rubin-style FITS header with SIP distortions and run the transform chain for that end-to-end. I had it write benchmarking tooling to compare performance of the polynomial distortions between C and Rust with different numbers of inputs to compare the performance of small numbers of points (C is relatively slow with those) versus large numbers of points (C uses extensive up-front setup that is optimized for large numbers of points).

Towards the end of March there was a significant drop off in code generation as the agent tied itself in knots trying to work on simplification logic and this resulted in a decision to step back from the port and re-assess (I had run out of tokens in two days). It also turned out that March 2026 was special in that OpenAI had allocated significantly more tokens to users of the new Codex app than they would subsequently allow towards the end of April, leading to a false sense of abundance that would turn out to be short-lived.

_images/starlink-astrs-proto-loc.svg

Fig. 2 Lines of Rust code as a function of time for the first Rust prototype. The gap indicates time not working on the port.#

Lessons Learned#

_images/simplify-passes.svg

Fig. 3 An example of how the AST simplify logic can work with multiple passes converting 8 mappings to 3. Note how it linearizes the mapping chain and then starts walking through from first to last. Every reduction in mapping count leads to more transform performance. The simplification engine is one of the places with the most complex logic and also is the easiest to misunderstand and end up implementing simpler algorithms if the test fixtures are not driving completeness.#

In about 10 days the port implemented about 15% of the C library’s logic, dominated by code for reading a Rubin FITS header and transforming the resultant FrameSet from pixel coordinates to sky coordinates and vice versa. Writing of FITS headers was not implemented but native plain text serialization was added for the supported subset of mappings.

The work did demonstrate one of the key motivations for the investigation. It has long been known that AST is extremely inefficient at transforming small numbers of points. This is because there are setup costs in the C before a transformation can trigger but those are not necessary in Rust. For the representative Rubin FrameSet a single point transform is 3- to 8x faster than the C. For large numbers of points in some cases with the PolyMap transformation the Rust is 1.2x faster going forward and 1.5x faster for the inverse direction, without using vectorization. Clearly some of these optimizations for the large scale PolyMap improvements could also be applied to the C, but it did provide solid evidence that adopting Rust would not lose performance. One key insight from this is that the ability to run tests and benchmarks directly against the C “oracle” is invaluable. Codex built a full oracle test and benchmarking harness and was able to iterate rapidly on the profiling because of this. Additionally it instrumented the C and Rust code to report exactly which trigonometry calculations were being triggered and could therefore prove that the C and Rust were doing the same number of calculations when they got the same answer, instead of the Rust getting the answer because of a heuristic. One ongoing lesson from this work is that agents will cut corners to solve the immediate problem even if that causes a later problem.

When this work started it was an open-ended port where guidance was given to the agent interactively but there was only the one guiding principle of porting everything. One decision I made early on was to record porting decisions in a single place (PORT.md), eventually having it be updated every time a decision was made or some problem was hit. This provided a valuable historical record but no thought was given as to how it affected the agent’s context – the file was being scanned by the agent at every step since it was required to keep it up to date and eventually that content dominated the context and token usage (this was before contexts in the millions were supported). In retrospect it would have been better to have this document be used as an active record of the current state of the port rather than it continuing to grow as new information becomes available. Having that file instead only include the active to do list for the port plus any decisions made in the port and any mistakes made that should not be repeated, would have given a more bounded document that could iteratively improve the quality of the port without overflowing the agent’s context.

There were a number of interesting problems encountered during the work:

  1. When looking at a dual-sideband FITS header the agent was not calculating the correct spectral values and tried multiple times to work out why there were differences, including optimizations involving irrelevant FITS header cards. Once the error was below 300 Hz it couldn’t make any improvement and was at risk of saying that was close enough. It turned out that the real problem was that there was a missing timescale conversion from UTC to TDB earlier in the chain.

  2. One of the key aspects of AST is its ability to simplify complex FrameSets into simpler forms by noting cases where two adjacent mappings can be combined (such as a forward mapping and an inverse mapping canceling out, or a ZoomMap and a MatrixMap being combined into a single MatrixMap). This is critical for performance since it minimizes the number of transforms to be calculated. When I asked Codex to add simplification I failed to notice that, even though it mentioned in the porting document that the C library has simplification logic per mapping, it was putting special-cased simplification logic in a single file. The only clue was firstly that it was having a harder and harder time matching the simplification logic from the C as it stacked more and more variations in one place, but also that the cmp_map.rs file was becoming significantly larger than the C original. This was an example where the agents are by default instructed to do the minimum work to satisfy the goal, and adding some specialist simplifications in the main file was less effort than setting up an entirely new multi-mapping infrastructure.

  3. At no point did I explain the AST__BAD magic value to the agent, and so it mostly failed to add support for it in the transform code path. This was possible because at the time there was no C test code that triggered the problem and so there was no motivation to implement it. Codex did eventually come up with a checksum approach for the oracle transform testing to allow it to do many transformations and quickly spot problems such as the missing AST__BAD support.

In addition to these issues there were some more serious examples of the agent faking test results.

  1. For the FITS header tests the agent knew that the idea was to read in the FITS header, convert it to a FrameSet and then write it out in the native plain text serialization and compare it with the oracle. The test succeeded because there was specialist code added that on read would recognize which FITS header it was, inject the serialized oracle form into an internal cache, and then on write it would write out the cached text. This is clearly the easiest way to pass the test but over the 10 days Codex was caught multiple times trying to cache a pre-canned answer so it could pass the test.

  2. In another example Codex rewrote the serialized form to make it look like the correct oracle form.

  3. At one point some test fixtures that were meant to be generated by the oracle were in fact generated by the Rust – a guarantee that the test would pass.

  4. On another occasion the Rust code started to special case different combinations of spatial and spectral FITS headers since it did not understand that in most cases parsing the spatial WCS is distinct from parsing the spectral WCS. It also created an entirely new class, TanSkyMap, specifically to handle one specialist FITS header.

  5. The degen1.ast example FrameSet is one of the trickiest to handle since it has a degenerate WCS with mismatched pixel axes. The Rust test code was failing to generate a simplified form that matched the C and so it had a special clause in the code specifically for this FrameSet to reorganize it into the required form. The workaround was explicitly turned off when reading similar example files that would be broken by the rewrite.

  6. Special-cased handling was also included for some forms of CAR headers.

There were also inefficiencies:

  1. When parsing a SIP FITS header, the code was creating a serialized native text form of the polynomial definition and handing it to PolyMap to be parsed rather than just using the direct PolyMap constructor.

  2. As mappings were added I was interested in knowing how the Rust compared to C with transform performance. This was too early in the process and driving the agent to make something faster led to it putting performance optimizations in the wrong places which then led to downstream confusion that built on those mistakes.

The fundamental learning from the prototype work was that test coverage of the C library is critical, as is the use of the oracle to provide immediate feedback to compare with the new code when there is a disagreement. At the time test coverage in AST was mainly handled through Fortran testing with no formal coverage reporting. This then provided the focus for the next phase.

Note

Key insights:

  1. Test coverage of the oracle library is critical.

  2. The oracle as source-of-truth always wins and should be consulted throughout the port.

Tests Are Important#

By April 2026 it was realized that there was some fundamental work that could be done on the C library that could improve the quality of the Rust port.

The first task was to simplify the build system. The AST C library uses the GNU Autoconf tooling but with Starlink extensions for handling some of the Fortran interfaces [Gray et al., 2005]. The core library is not in itself complicated to build, but the Git source has no means to build it automatically without having access to the Starlink software distribution. For the prototype I got around this by teaching Codex how to build the oracle based on it working out how the Python bindings were built. This was not the most efficient approach and so in April 2026 I used Claude Opus 4.6 to build an entire parallel CMake build system that would build the C library (not the Fortran bindings) and run the C tests. I also methodically converted the Fortran tests to C and added coverage reporting into CI so I could determine the current state of the test suite. For one of these Fortran test files (about 1300 lines of Fortran) I had run out of Claude tokens and switched to Gemini Pro 3.1 – it took a couple of hours and didn’t make much progress. When I had access to Claude it looked at the code, determined that the code was a very naive direct Fortran-to-C port, and decided it was easier to rewrite it from scratch, which it did in ten minutes. Additionally, Gemini had read the instruction that there should be no new warnings introduced so it decided to use #pragma to turn off the warnings in its new code.

This work was focused on determining the current code coverage as a whole and not specifically the parts that I wanted to focus on for the initial port, and so included coverage for the plotting subsystem. Since the Fortran plotting tests required PGPLOT I added a new graphics backend for PLplot and additionally Claude wrote a native SVG plotting backend which was very useful since Claude can read SVG outputs natively and reason about their contents, allowing it to iterate rapidly to ensure that the SVG was rendering correctly. When this code was merged the code coverage for the core library code was approximately 50%. Adding GitHub Actions CI also enabled me to easily do sanitizer builds, which immediately found a couple of latent buffer overruns that had been around for a long time.

Once that was in place the next step was to create more example FITS headers. For this work Claude was tasked with improving the coverage solely of the FITS handling file. It was able to read the code and then attempt to create a FITS header targeting a specific line. The FITS header handling is one of the most esoteric parts of the AST source since it encodes many historical variants of FITS headers, many of which are not in wide circulation today, but it was felt that a port that could only read current FITS WCS variants would not be a real port, and I could not abandon the historical support (the AST library is used by the SAOImageDS9 image viewer specifically for this reason). This work added 90 new FITS headers to the test fixtures, with the code coverage for the FITS handler increasing to 75% – the remaining coverage was difficult to establish since much of it involves correctness tests that are acting as safety nets confirming a situation that is already handled earlier in the chain.

This work provided the foundation for the next porting attempt.

Starting Again From Scratch#

Once I had a better build and test story for the C library, by late April 2026 I decided to make another attempt at doing the port from scratch. By this time Claude Opus 4.6 was the current model and in addition to a $20/month team account I had access to a Claude API key through the DOE SLAC National Accelerator Laboratory near Stanford. Since Rubin is partially funded by DOE I was allowed to make use of their new infrastructure based on Amazon Bedrock since SLAC was instructed to investigate ways in which LLMs can be used in research activities. For FY26 SLAC is absorbing the cost of the API keys as part of the research experiment so Rubin was able to make use of this without being charged.

This time I decided to start by planning what I thought the port should look like in terms of memory management and mutability and how traits should relate to C methods. As was done previously the wcsLib, ERFA, and PAL source code was copied unchanged with Rust shims on top.

Work started on 2026-04-27 with Claude Opus 4.6. For planning I used the Superpowers plugin, and this was particularly helpful in iterative specification development and then building and documenting implementation plans. It defines a full software development process involving using subagents (usually of a lesser model) to implement the plan, and then review of that work by the main agent. This can give very nice results although subagents tend to use far more tokens than using the main model, even if a lesser model is doing the work.

Work progressed reasonably well through August 2026 with the main impediment to progress being token starvation. It was possible to hit my monthly SLAC API key ceiling in the first week. It was noticeable how Fable (for some limited testing) and Opus 5 (compared to Opus 4) were able to come to solutions more quickly than the older models, so even though they were nominally more expensive per token the net cost per feature can be less.

_images/starlink-astrs-loc.svg

Fig. 4 Lines of code as a function of time for the second Rust port.#

Lessons Learned#

In the prototype I attempted to build a C ABI incrementally in order to ensure that C semantics could work. On the second attempt I deliberately punted the C ABI until the very last stage once full functionality was ported. At this time it is unclear whether this was the right approach since it is much harder to change direction after a full implementation. A simple demonstration of a FrameSet and a ZoomMap transforming a number from the C side would give more confidence in the approach specified in the spec but nothing has yet been tried. Even more importantly, the ownership model in Rust where adding a mapping to a CmpMap transfers ownership to the CmpMap is not how the C behaves where adding a mapping to a CmpMap clones the mapping – once inside the CmpMap the mapping is marked as immutable so there is no change in core behavior, but there is a change at the edges where in C if you try to change the state of a mapping after it has been added to a CmpMap you get an error but if the Rust port has done a deep copy your modification would look like it is working but would have no effect on the CmpMap.

As the port grew in size it again suffered from lack of discipline as interesting side tracks were chased. This led to the FITS channel handling being implemented before all of the core mappings (especially TimeFrame) were complete. Lacking TimeFrame meant that time conversion logic became embedded in the FITS handler rather than in its correct place.

Somewhat embarrassingly this second port made a similar mistake regarding simplification logic and where it was located. In late May I went back to the AST library and decided to methodically generate hundreds of CmpMaps in order to maximize code coverage and provide a true reference corpus of the simplification logic. This immediately led to vast numbers of failures on the Rust side including a critical discovery that the CmpMap flattening in Rust was using a different (but mathematically equivalent) algorithm to the C code – transforms were working fine before and after simplification but the mappings had a different shape when serialized. Fixing that took a while to recover with many commits porting C algorithms and reorganizing all the related MapMerge logic into each mapping where it should have always resided. Even by late August with 400 test fixtures passing it was discovered that some parts of the simplification chain were in the wrong place and other places were missing with workarounds. This was discovered with a full audit comparing the C and Rust side. It was even discovered that some bespoke logic had been included in the serialization layer to allow a test to pass even though the simplification logic was not fully accurate. Given that the transform logic was working correctly and the serialization output was incorrect in an edge case, I still needed to match the C logic to give me confidence I had not missed something critical. Even worse, whilst trying to recover from the earlier mistakes and instrumenting the entire simplification engine I realized that very early on the port had written a simplified() API that corresponded to the internal vtable implementation but which incorrectly included some map_merge logic when it should not and crucially was called in much of the code instead of the public simplify() method. This was the root cause of the simplification discrepancies since only simplify() stamped the IsSimple flag and calling the internal method bypassed that flag and led to premature simplifications or redone simplifications. Furthermore, having some logic in simplified() instead of map_merge meant that the CmpMap simplification could not be reproduced. Recovering this dominated the late August work.

Because there was no internal tracking of port status in any committed document there were some cases where some classes were added (such as the regions) but only implemented a subset of the internal C logic. Since nothing was tracking the missing logic there were cases in other parts of the code that worked around the absence rather than understanding that the real fix was to fix it properly.

Early on there were hundreds of test fixtures failing and so those tests were made optional. This was fine except it was easy to forget that they were failing when all tests were green. The Rust test infrastructure does not include an “expected fail” state either that could let me know if a fix elsewhere led to a test passing that was not expected.

Many tokens were spent trying to generate the correct invert flags in the FrameSet serializations. These flags record the original state of the invert flag in the mapping since the invert state is owned by the CmpMap but because the C clones the mapping it needs to record that original state to make sure that the user’s clone does not change. The Rust ownership model does not need to track this and since the design spec did not realize the significance of these flags in the C code it took many iterations and reverted experiments to get them to match up with the serialization. This is clearly a C implementation detail but has to be tracked for byte-exactness and it took too long to understand its true meaning.

Porting as Bug Finding#

During the porting work there were times when the agent would be unable to reproduce a fixture or a test and determine that there was a possible bug in the upstream C library itself. As of August 2026 I had found numerous bugs and fixed them. Some of them required updating of reference fixtures and so there was a virtuous cycle of finding a bug, reporting a bug, fixing a bug, merging the upstream fix, and syncing upstream with the port.

The port discovered and led to fixes of the following bugs. I include them here to give an idea of the different types of issues encountered. Most of them were very small fixes.

  • RefRA/RefDec were round-tripped through a formatted string at default Digits, so spectral VELOSYS drifted; now formatted at 20 digits.

  • MatrixMap MapSplit advanced its matrix pointer only past non-BAD elements, so it read uninitialized memory and split results were nondeterministic.

  • ScalePolyInputs’ inverse-coefficient branch indexed off a stale pointer: an out-of-bounds read.

  • DATE-OBS fractional seconds were parsed with "%d", dropping leading zeros: .087 became 0.87 s, a 783 ms epoch error.

  • SIPIntWorld read the output-axis count from astGetNin instead of astGetNout; a one-word typo.

  • SpecMap’s velocity helpers cast RefRA/RefDec to float, discarding ~9 significant digits inside every LSRK/LSRD/LG/GAL correction.

  • CLASSFromStore multiplied the spectral CDELT by specfactor twice, so FITS-CLASS output carried a squared unit scale.

  • astDat’s pre-1972 inverse divided by 1.02592 instead of 1 + b/SPD (off by a seconds-per-day units factor), leaking ~164 ms per UTC→TAI→UTC round trip for 1960s epochs; the same commit fixed a transcribed 1966 intercept (4.2131700 → 4.3131700).

  • astAxAngle’s degenerate-case nudge indexed its coordinate array with the loop variable rather than the axis argument, returning BAD instead of a valid angle.

  • astRate seeded its numerical step with an absolute 1.0, so it returned BAD at x0 = 0 for any Mapping with a narrow domain (such as ChebyMap).

  • astRate’s zero-range confirmation read the range array at the loop counter instead of the stored index, confirming the wrong axis.

  • PermMap::Equal skipped comparing the referenced constants when both index entries matched, so PermMaps with different constant values compared equal.

  • SpecFluxFrame’s astEqual reported two identical frames as unequal.

  • Negative grism interference orders were corrupted on read (−2 became −1, −1 became 0) due to negative floats rounding in the wrong direction.

  • The (int)(x+0.5) idiom rounds every negative value toward zero; fixed in ConvertValue and then swept tree-wide across 70 sites in 16 files, including the WCSAXES axis count.

  • astMapIterate never terminated on a KeyMap with SortBy set: an infinite loop, latent because the only in-tree caller never sets it.

  • astMapRename on a locked KeyMap deleted the entry and then refused to re-add it, so the data was simply lost.

  • Out-of-range and NaN double-to-integer conversion in ConvertValue was undefined behavior; now range-checked with a defined result.

  • Numeric-string overflow into %d was UB, and glibc silently wrapped rather than failing; now detected.

  • A zero-length vector entry was dumped by reading an Entry1X as an Entry0X, writing pointer bytes as the value and null-dereferencing for one type; now rejected at astMapPut1<X>.

  • astMapGetElem<X> reported success on an undefined entry while writing nothing to the caller’s buffer, unlike astMapGet0<X>/astMapGet1<X>.

  • LoadKeyMap injected a spurious KyCas = 1 on every re-dump, because KeyCase’s unset sentinel is -1, not -INT_MAX.

  • astLoadKeyMap applied MpLck before loading any entry, so a locked non-empty KeyMap could never be read back at all.

  • LoadKeyMap read a Mem%d card that DumpEntry never writes, resetting the member counter so every loaded entry got member 0.

  • astMapCopyEntry normalized the key with the source KeyMap’s KeyCase, for both the destination lookup and the stored key rather than using the key case of each independently.

  • EncodeFloat stripped a redundant exponent zero by shifting the prefix right rather than closing the gap, so any value wider than 20 columns (e.g. FitsDigits=17) was emitted one column right of every other card, running past column 30. Malformed FITS output.

  • Legacy CDjjjiii cards were read (and so consumed) under every encoding but stored only under FITS-IRAF, silently losing the rotation matrix elsewhere.

  • SpecTrans marked only the first copy of a repeated keyword while WcsFcRead swept all of them, so reading a two-HDU header left a self-inconsistent partial WCS behind; MJD-OBS survival also depended on whether an unrelated DATE-OBS sat alongside it.

There were also fixes that enforced stability in the native serialization format. Whilst the serialization format was historically meant to be treated as a private format understood solely by the C library, validating the Rust port requires that the output is predictable and can be reasoned about.

  • WcsNative built one PermMap, used it in two stages, and inverted it in between. Since astCmpMap clones rather than copies, the stage-1 CmpMap serialized a vestigial Invert = 1 on its child that the transform path ignores.

  • KeyMap dump ordering was unpredictable. Dump walked hash buckets and LoadKeyMap head-inserted on reload, so keys sharing a bucket swapped places on every dump/load cycle and a KeyMap’s serialization never converged. Now dumped in key order.

  • IsSimp tag placement in dump depended on internal aliasing. A single Mapping.flags bit meant both “dump IsSimp = 1” and “do not simplify again”, and astSetInvert cleared it as a side-effect, in particular as happens when astEqual is called. Which interior CmpMap node ended up carrying the tag therefore depended on shared-pointer aliasing history during simplification, with no structural rule a re-implementation could follow.

Fixes were also made as part of the enhancements to the test coverage, but those are not directly attributable to the Rust port.

Port Timeline and Status#

_images/starlink-port-interleave.svg

Fig. 5 A timeline for the work described in this document as of mid-August 2026. You can see how the prototype triggered work in the C library to update the build system and add test coverage and that later work on the port resulted in separate work on the C library (bug fixes and additional test fixtures). The spike in the AST commit history in May is adding many more simplification example test fixtures.#

The timeline for the port is shown in the figure above and demonstrates the different phases of the project and how the upstream test improvements and bug fixes interleaved with the core porting work. This timeline includes most of the mappings and all the frames and non-STC regions. It also includes Channel and FitsChan but none of the other channels (YamlChan, XmlChan, Moc and MocChan, StcsChan). It does not include Plot, Plot3D, any grf backend, Table and FitsTable, IntraMap, or XphMap. Also missing is the core mapping functionality of astResample, astRebin, astRebinSeq, astTranGrid and astQuadApprox. These missing features account for about 30% of the code volume of the C library. There is no C or Python interface and there have been minimal performance optimizations – the second port is currently slower than the prototype in many cases.

From a distance it looks to the naive user that stacking some transforms together should be relatively straightforward. Both the ports demonstrate, in their initial velocity, that setting up some core mappings in a FrameSet is the quick part. The complexity comes in all the optimizations and edge cases. In particular the simplification system is fundamental to core performance – a user should be able to construct the mappings in a way that is obvious to them and let the library convert them to an optimized but equivalent form. The frame conversion system is also important and complex. Porting a library of this size shows that the 80% is relatively straightforward but the final 20% is where all the edge cases and optimizations are.

Recommendations#

The key takeaway from this work is that your language port is only as good as the test coverage of your original code. The coding agent will try to take the easiest path to generating the code from your prompt. You can ask it to be faithful to the source language but you can’t be sure it is really going to do that unless there are tests that demonstrate exactly what you need. For AST this is made more explicit by the addition of hundreds of test fixtures involving serialized FrameSets and Mappings as both native AST serializations and FITS headers. Adding test coverage to the source library before you start the port, especially for edge cases, pays dividends many times over once you start on the port. You can use a coding agent to add the tests.

  • Always write a plan first and refer to that plan in your CLAUDE.md or equivalent so that the plan is checked against implementation. Allow the plan to be updated based on lessons learned.

  • Keep a running ledger of open work so that you do not continually have to scan the C and Rust source to check what has been ported.

    • You will want to port every test so for auditing purposes annotate in your new tests where the test came from originally.

    • Consider having a TODO list of every test that you can then mark off as the port progresses.

  • Correctness and completeness are more important for a port than performance. Do not start optimizing until you are passing every test. Even if you have code coverage for the small area you are optimizing you make it harder later on to reason about whether the Rust version truly matches the C approach and sometimes the optimization can shift logic around that confuses a later code addition that relied on the earlier form.

  • You do not want to end up with a line for line port of the source library to the new language. You want to ensure that the algorithms are ported (with annotations) whilst making use of new language features. For example, for the Rust port I used the standard Option and Result approach to status handling rather than inherited integer status. The AST library is particularly well-suited to porting since the original authors were very careful to explain all their decisions as detailed comments and to carefully separate each chunk of logic (there are more comment lines than code lines). This allowed the Rust port to easily point at distinct blocks of code in the original.

  • Ensure that your coding standards cannot be bypassed. Use pre-commit / prek hooks to capture all code linting such as ruff and numpydoc in Python and clippy in Rust. Every gate you can apply to commits forces the agent to follow your guidelines in a far more robust way than making a note in the CLAUDE.md file ever would. In Python use as many ruff check options as you can.

  • Some coding practices do have to be included in the CLAUDE.md file but remember that this is guidance and not treated as a hard rule; the prompt and history will all be processed along with the CLAUDE.md. Negative directives are weaker than positive directives. For Python, agents have a tendency to prefer to add imports locally rather than at the top of the file so consider adding an instruction to always put imports at the top unless there is an import cycle.

  • For the AST port it was critical to strongly declare that the C algorithms were the source of truth and were to always be used as is and to only use heuristics in the Rust if the same heuristic was present in the C. The C library is the “oracle” and you have to insist that whenever there is a difficulty in getting a Rust test to pass that the oracle be checked and instrumented.

  • In some cases there will be bugs in your original library and these will be noticed by the coding agent. Sometimes the agent will not tell you about them (such as triggering a SEGV in an oracle investigation) so you have to ask. Sometimes the agent will realize there is a bug in the original but code that bug into the port. I eventually added a standard instruction noting that bugs are possible in the original and should be recorded in a separate document.

  • Sometimes the agent will flail around trying to work out what is going wrong. Remember that you have an oracle. Suggest to the agent that it instruments the original library so it can trace exactly what code is involved for a passing test. This helped many times in the port of the AST library for mapping simplification verification and for performance validation such as ensuring the same number of trigonometric calls occur in both the original and the port.

  • If the target language has native tracing/debugging/logging facilities embed those features into the port as you go. If you do not do this you will notice that the agent is continually adding and removing debugging instrumentation. Even better if you can add similar facilities to the original library to allow direct comparison. Agents can work much faster if they can turn on trace logic on both sides and immediately see the shape of the difference without having to cycle through phases of code churn each step of the way.

When porting from one language to another you have to decide whether “byte exact” is important or if you are targeting test completeness. The former is much harder than the latter but is required if this is a mission critical core library. Once you are byte exact, you are then able to focus on performance optimizations.

Conclusions#

At the time of writing it is clear that a Large Language Model coding agent such as Claude Opus/Fable or ChatGPT 5.6 Sol is fully capable of porting large libraries to different languages so long as the original library is well tested. The test suite, planning, and a reference oracle mitigate many of the problems encountered when using LLMs to write large amounts of new code that has no such boundary conditions. At Rubin the observatory software team is experimenting with using agents to convert LabVIEW to Rust. The LSST Science Pipelines have 150,000 lines of C++ code in libraries and it is now clear that converting that to Rust is a plausible enterprise if the project wishes to do so. Most of those pipelines libraries are significantly less complex than the AST library.

Acknowledgments#

I thank David Berry and Jim Bosch for their feedback on the initial design spec and for answering numerous questions. In particular David also was tasked with reviewing some of the giant AST pull requests. I also thank Michael Shehadeh from SLAC IT for his help with the Claude configuration and API key at SLAC. The project would not have been possible without the generous allocations of tokens from SLAC IT.

References#

[1] (1,2)

James F. Bosch. Challenges serializing Rubin Observatory data products. Data Management Technical Note DMTN-339, NSF-DOE Vera C. Rubin Observatory, June 2026. URL: https://dmtn-339.lsst.io/.

[2]

M. R. Calabretta, F. G. Valdes, E. W. Greisen, and S. L. Allen. Representations of distortions in FITS world coordinate systems. 2004. URL: https://fits.gsfc.nasa.gov/wcs/dcs_20040422.pdf.

[3] (1,2)

Saman Dehghan, Tianran Sun, Tianxiang Wu, Zihan Li, and Reyhaneh Jabbarvand. Translating large-scale c repositories to idiomatic rust. 2025. URL: https://arxiv.org/abs/2511.20617, arXiv:2511.20617, doi:10.48550/arXiv.2511.20617.

[4]

Ibrahim A. Elgendy, Yogesh K. Dwivedi, Mohammed A. Al-Sharafi, and others. Responsible vibe coding: architecture, opportunities, and research agenda. Journal of Computer Information Systems, 0(0):1–19, 2026. URL: https://doi.org/10.1080/08874417.2026.2621186, doi:10.1080/08874417.2026.2621186.

[5]

Ali Reza Ibrahimzada, Kaiyao Ke, Mrigank Pawagi, Muhammad Salman Abid, Rangeet Pan, Saurabh Sinha, and Reyhaneh Jabbarvand. Alphatrans: a neuro-symbolic compositional approach for repository-level code translation and validation. Proc. ACM Softw. Eng., June 2025. URL: https://doi.org/10.1145/3729379, doi:10.1145/3729379.

[6]

Steve Klabnik, Carol Nichols, and Chris Krycho. The Rust Programming Language. No Starch Press, 3 edition, March 2026. ISBN 978-1-7185-0444-8. URL: https://doc.rust-lang.org/book/.

[7]

John Lakos, Vittorio Romeo, Rostislav Khlebnikov, and Alisdair Meredith. Embracing Modern C++ Safely. Addison-Wesley Professional, 2021. ISBN 978-0-13-738035-0.

[8]

Scott Meyers. Effective Modern C++: 42 Specific Ways to Improve Your Use of C++11 and C++14. O'Reilly Media, November 2014. ISBN 978-1-491-90399-5.

[9]

John Parejko and Russell Owen. WCS and Distortion Requirements and Existing Options. Data Management Technical Note DMTN-010, NSF-DOE Vera C. Rubin Observatory, May 2016. URL: https://dmtn-010.lsst.io/.

[10]

Advait Sarkar and Ian Drosos. Vibe coding: programming through conversation with artificial intelligence. In Proceedings of the 36th Annual Conference of the Psychology of Programming Interest Group (PPIG 2025). September 2025. doi:10.48550/arXiv.2506.23253.

[11]

Doug Tody, Lindsey David, and Frank Valdes. Tnx convention for representing fits image distortions. 2009. URL: https://fits.gsfc.nasa.gov/registry/tnx.html.

[12]

Frank Valdes. TPV convention for Representing FITS Image Distortions. 2012. URL: https://fits.gsfc.nasa.gov/registry/tpvwcs.html.

[13]

Frank Valdes. ZPX convention for Representing FITS Image Distortions. 2012. URL: https://fits.gsfc.nasa.gov/registry/zpxwcs.html.

[14]

Rich Yonts. 100 C++ Mistakes and How to Avoid Them. Manning, 2025. ISBN 978-1-63343-689-3.

[15]

D. S. Berry. Developments in the Starlink AST Library - an Intelligent WCS Management System. In Francois Ochsenbein, Mark G. Allen, and Daniel Egret, editors, Astronomical Data Analysis Software and Systems (ADASS) XIII, volume 314 of Astronomical Society of the Pacific Conference Series, 412. July 2004.

[16]

David S. Berry, Rodney F. Warren-Smith, and Tim Jenness. AST: A library for modelling and manipulating coordinate systems. Astronomy and Computing, 15:33–49, April 2016. arXiv:1602.06681, doi:10.1016/j.ascom.2016.02.003.

[17]

M. J. Bly, D. Giaretta, M. J. Currie, and M. Taylor. Starlink Software Developments. In H. E. Payne, R. I. Jedrzejewski, and R. N. Hook, editors, Astronomical Data Analysis Software and Systems XII, volume 295 of Astronomical Society of the Pacific Conference Series, 445. January 2003.

[18]

M. R. Calabretta. Wcslib and Pgsbox. Astrophysics Source Code Library, record ascl:1108.003, August 2011. ascl:1108.003

[19]

M. R. Calabretta and E. W. Greisen. Representations of celestial coordinates in FITS. A&A, 395:1077–1122, December 2002. arXiv:astro-ph/0207413, doi:10.1051/0004-6361:20021327.

[20]

N. Gray, T. Jenness, A. Allan, and others. Porting the Starlink Software Collection to GNU Autotools. In P. Shopbell, M. Britton, and R. Ebert, editors, Astronomical Data Analysis Software and Systems XIV, volume 347 of Astronomical Society of the Pacific Conference Series, 119. December 2005.

[21]

T. Jenness and D. S. Berry. PAL: A Positional Astronomy Library. In D. N. Friedel, editor, Astronomical Data Analysis Software and Systems XXII, volume 475 of Astronomical Society of the Pacific Conference Series, 307. October 2013.

[22] (1,2)

Rubin Observatory Science Pipelines Developers, Yusra AlSayyad, Tim S. Axelrod, and others. The LSST Science Pipelines Software: Optical Survey Pipeline Reduction and Analysis Environment. Project Science Technical Note PSTN-019, NSF-DOE Vera C. Rubin Observatory, July 2026. URL: https://pstn-019.lsst.io/, doi:10.71929/rubin/2570545.

[23]

D. L. Shupe, Mehdrdad Moshir, J. Li, D. Makovoz, R. Narron, and R. N. Hook. The SIP Convention for Representing Distortion in FITS Image Headers. In P. Shopbell, M. Britton, and R. Ebert, editors, Astronomical Data Analysis Software and Systems XIV, volume 347 of Astronomical Society of the Pacific Conference Series, 491. December 2005.

[24]

R. F. Warren-Smith and D. S. Berry. World Coordinate Systems as Objects. In Rudolf Albrecht, Richard N. Hook, and Howard A. Bushouse, editors, Astronomical Data Analysis Software and Systems VII, volume 145 of Astronomical Society of the Pacific Conference Series, 41. January 1998.

[25]

P. Škoda, P. W. Draper, M. C. Neves, D. Andrešič, and T. Jenness. Spectroscopic analysis in the virtual observatory environment with SPLAT-VO. Astronomy and Computing, 7:108–120, November 2014. arXiv:1407.1765, doi:10.1016/j.ascom.2014.06.001.

Appendix: Initial Port Design Spec#

This is the initial design spec for the overall shape of the port from C as developed iteratively with the Superpowers plugin brainstorming mode on 2026-04-20 with input from David Berry and Jim Bosch. Some things have changed as the port evolved and an updated spec can be found in the Git history.

Revision history#

  • 2026-04-27 (rev 5): tighten test-coverage scope after the upstream sync of Starlink/starlink-ast (which added batch 13 and more fixture coverage):

    • Full ast_tester migration is now a hard project goal, not a “ported class-by-class” aspiration. Every .c test program in ast_tester/ must have a Rust counterpart in ast-core/tests/, and every reference fixture (.head, .ast, .native, .fits-*, .attr, .box, .fattr, .simp, .current, .moctohtml, teststc_eg*, …) must be copied into the Rust test corpus and exercised by the corresponding Rust test or harness.

    • Three regression harnesses get explicit Rust analogues: wcsconverter (drives the .fits-* encoding round-trips), simplify (drives the .simp and .current references), and the plot/grid smoke matrix (drives .head + .attr + .fattr

      • .box per fixture).

    • Tier-1 migration is paced with the implementation (each delivery step ports the tests its classes enable). Step 11 is the completion gate that confirms 100% migration, not the step at which the bulk of migration work happens.

  • 2026-04-27 (rev 4): further PR feedback and a project layout decision:

    • Repository layout reworked. The Rust port lives in its own fresh git repository (starlink-astrs) with Rust-only commit history. The new repo takes ownership of the vendored C dependencies (pal, erfa, wcslib) by copying them in-tree at vendor/, so the everyday build (cargo build, cargo test) works on a fresh clone with no submodule initialisation. A submodule at reference/starlink-ast/ brings in the upstream C library and is needed only by the optional oracle test tier (Rust-vs-C comparison) and the FFI parity tier (existing C ast_tester programs against ast-capi). Replaces the rev-3 layout’s four separate vendor/ directories.

    • Axis indexing switched to 1-based in the Rust API. Rev 3 proposed 0-based with CFFI translation; per @timj that produces a silent footgun (users reading AST docs see Label(1) and would reach for set_label(1, …) and quietly target the wrong axis) and requires unwanted translation at the Native-serialisation layer (Lbl1/Lbl2 tokens). 1-based for axis attributes and for FrameRef::Index matches AST docs, the Native format, and the C API throughout.

    • TimeFrame jd1 + jd2 precision added as a forward-looking note inside the inherited-limitations section: a future TimeFrame attribute supplying a fixed offset MJD would let the internal transforms use ERFA’s two-double-precision API without changing the public f64 interface.

  • 2026-04-24 (rev 3): further review decisions from lsst-dm/starlink-astrs#2:

    • Open questions closed:

      • generics-over-Frame is dropped (not appropriate for this port’s goals, per @TallJimbo).

      • enum-based Mapping dispatch is dropped from the spec; may be introduced later as a pure internal optimisation.

      • Rust-backed Python binding is out of scope and removed from the open-questions list.

    • test_attr(name) and clear_attr(name) by string are removed from the Rust public API. Only typed per-attribute test_* / clear_* methods remain. Per @TallJimbo: stringly-keyed attribute access is a __getattr__-style pattern we should avoid without a reason.

    • Axis-indexed attributes (Label(1), SkyRef(2), …) get an explicit section specifying the typed-accessor shape.

    • Ownership of frames within a FrameSet made explicit: DynFrame is a single-owner Box; add_frame transfers ownership.

    • wcslib promoted from “non-goal, revisit later” to planned for a Rust port later in the implementation. Motivation: the vendored snapshot is ~20 years old; upstream has since diverged significantly and gained vectorised APIs that WcsMap could use if we control the implementation. pal / erfa remain vendored C (simple enough to port later if a perf win motivates it).

    • Added a “Known inherited limitations” section noting the A&C-paper items (radians-vs-degrees for SkyFrame; coordinate-system vs domain conflation; double-precision time axes) that cannot be fixed while preserving the C ABI.

  • 2026-04-24 (rev 2): incorporates review feedback from lsst-dm/starlink-astrs#2. Major changes:

    • Mutability model flipped from immutable+Arc (v1 option A) to owned-values with &mut self mutation (v1 option B). Driven by SCUBA-2 / SMURF evidence of per-time-slice Epoch and SkyRefIs mutation on SkyFrame, and plotting’s tight-loop Digits updates.

    • Trait hierarchy flattened: Frame and Mapping are now independent sibling traits. FrameSet and Region implement both directly. Converged position between @TallJimbo and @dsberry.

    • New first-class section on FrameSet integrity and the matching Region integrity mechanism — attribute mutation on the composite propagates to internal mappings / defining positions.

    • Coordinate-array types split into PointSet (frame-unaware, ndarray-backed) and PointList (frame-aware, a Region), matching the existing AST distinction.

    • Operator-overload sugar for mapping composition removed; then / alongside verbs only.

    • String-typed get_attr / set_fmt removed from the Rust public API and relegated to the CFFI shim; typed accessors plus test_attr and clear_attr remain.

    • Object demoted from public trait to an implementation-detail super-trait, per @TallJimbo’s observation that the public interface cannot actually be expressed on the bare Object methods.

    • Object::copy() removed — Clone is the deep copy.

    • Explicit semantics for adding a FrameSet to another FrameSet.

    • CFFI handle story updated for the owned-value world (Arc<RefCell<T>> at the handle boundary).

  • 2026-04-20 (rev 1): initial design.

Background#

The Starlink AST library is a ~100 kLOC C library providing world coordinate system (WCS) manipulation for astronomical data: FITS-WCS parsing, celestial and spectral frame transformations, graphical output, and associated utilities. It is consumed widely in the Starlink application suite, by pyast, and by numerous third-party astronomical codes. The current implementation is a hand-rolled object system with ~50 Mapping subclasses, a Frame / FrameSet hierarchy built on top of Mapping, Channel-based serialisation (Native / XML / YAML / FITS / STC-S / MOC), and Plot / Plot3D graphics driven through a link-time plugin system.

This document is the design spec for a full native Rust port of the AST library, exposing an idiomatic Rust API and a thin CFFI shim that preserves full binary ABI compatibility with the existing ast.h. It is deliberately a patterns-only spec: it commits to the overarching trait shape, ownership model, error conventions, serialisation architecture, plugin traits, and FFI strategy, with worked examples for representative classes. Per-class method-level design is deferred to the implementation plan.

Goals#

  1. A pure-Rust implementation of AST with an idiomatic Rust API — traits, Result-based errors, owned-value mutation via &mut self, strongly-typed attributes, ndarray-backed coordinate data.

  2. A cdylib shim (ast-capi) that preserves full binary ABI compatibility with today’s libast.{so,dylib,a}: existing applications link without recompile, without relink, without header changes.

  3. Byte-for-byte round-trip compatibility with every existing AST-serialised document (Native, XML, YAML, FITS-WCS, FITS-Native-encoded, STC-S, MOC).

  4. Full migration of ast_tester/ to native Rust. Every C test program (~53 .c files) becomes an idiomatic Rust integration test under crates/ast-core/tests/. Every reference fixture currently shipped alongside the C tests — .head (132), .ast (12), .native, .simp, .current, .fits-{wcs,iraf,pc,aips,aips++,class,dss}, .attr, .fattr, .box, .moctohtml, teststc_eg*, etc. — is copied into crates/ast-core/tests/fixtures/ and exercised by the matching Rust test or regression harness. The three large regression harnesses (wcsconverter, simplify, plot/grid smoke matrix) are reimplemented in Rust as test binaries that drive the same fixture sets. Reaching 100% migration is a delivery milestone (step 11 of the indicative order).

  5. All 107+ existing CMake ctests (107 default + 20 PLplot-conditional

    • 1 stress) pass against the Rust-built libast via the CFFI shim.

  6. Performance parity (within 10%) against the current C library on representative transform, simplification, and serialisation workloads. Hot-path constraint: attribute mutation (notably Epoch and SkyRefIs on SkyFrame) must be O(1) with no allocation and no locking, to preserve SCUBA-2-style per-time-slice call patterns.

Non-goals#

  • Porting pal / erfa to Rust. These are small and community-trusted; they remain vendored C and are built via cc from ast-core’s build.rs. Future perf work may motivate a port, but it is not planned here.

wcslib is in a distinct position: the vendored snapshot is ~20 years old and has diverged from upstream too far for the upstream to be adopted directly. A Rust port of the vendored fork is planned as one of the later implementation steps (step 6a in the delivery order), both to remove the Rust↔C boundary crossing on the transform hot path and to expose upstream’s later-added vectorised APIs to WcsMap.

  • Exposing the internal AstObject struct layout to external C callers. AstObject* remains opaque, as it is today.

  • Supporting a locking-based multi-threaded mutation story through the Rust API. Value-typed objects are Send but not Sync; ownership is exclusive within a thread. The CFFI shim preserves the C thread-safety story via its own handle registry.

  • A Python binding driven off the Rust types. pyast calls through the C ABI today; it continues to do so unchanged after the port. A future Rust-backed Python binding is possible but is neither a goal nor a constraint on this spec.

Approach#

Repository and workspace layout#

The Rust port is its own fresh git repository (starlink-astrs), not a branch or subdirectory of the existing C starlink-ast repo. This keeps the Rust commit history clean and uncluttered by the 20+ years of C history.

starlink-astrs/                # new fresh repo, Rust-only history
├── .gitmodules                # only for the optional reference submodule
├── Cargo.toml                 # workspace root
├── README.md
├── crates/
│   ├── ast-core/              # pure-Rust idiomatic API (rlib)
│   ├── ast-capi/              # cdylib; re-exports ast.h symbols
│   ├── ast-plugins/           # optional Grf / Grf3d backends (PLplot, SVG, logging)
│   ├── ast-ffi-tests/         # builds ast-capi, runs C ast_tester programs
│   ├── ast-oracle-tests/      # links ast-core + reference libast.a
│   └── ast-oracle-bench/      # criterion benches: Rust vs C on matched workloads
├── vendor/                    # copied in-tree, owned by THIS repo
│   ├── pal/                   #   small astrometry routines (was AST's pal/)
│   ├── erfa/                  #   ERFA snapshot (was AST's erfa/)
│   ├── wcslib/                #   AST's wcslib fork, pre-port (deleted at step 6a)
│   └── README.md              #   provenance notes: where each came from, what was changed
└── reference/                 # optional, populated only for tests
    └── starlink-ast/          # git submodule -> Starlink/starlink-ast at a pinned commit
                               #   used by ast-ffi-tests (its ast_tester C programs)
                               #   and ast-oracle-tests (its libast.a as the oracle)

Two key principles:

  1. The everyday build does not need the submodule. A fresh git clone + cargo build + cargo test -p ast-core works without any submodule initialisation. The vendored C sources in vendor/ are committed in-tree, owned by this repo. This matters for packaging (cargo publish, distribution tarballs) and for contributors who don’t care about the C parity tests.

  2. The submodule is opt-in for parity testing. Running tier-3 FFI tests (cargo test -p ast-ffi-tests) and tier-4 oracle correctness tests (cargo test -p ast-oracle-tests) requires git submodule update --init reference/starlink-ast. CI runs them; contributors who want them enable the submodule explicitly.

Notes:

  • vendor/pal, vendor/erfa, vendor/wcslib are copies of what the AST C library currently bundles, not git submodules to upstream projects. We take ownership: AST-specific symbol-renaming, patches, and dynamic-array changes are preserved verbatim during the copy. vendor/README.md records provenance (source repo, commit hash, date copied, list of AST-specific local modifications) so the history is recoverable without git.

  • ast-core’s build.rs compiles vendor/pal/, vendor/erfa/, and (until step 6a) vendor/wcslib/ via the cc crate, using the same symbol-renaming conventions palwrap.c / pal2ast.h / erfa2ast.h use today.

  • ast-oracle-tests and ast-ffi-tests add the submodule path to their build dependencies. If the submodule is not initialised, those crates are skipped (with a warning) rather than failing the workspace build.

  • After step 6a (wcslib port), vendor/wcslib/ is deleted from the Rust repo. vendor/pal/ and vendor/erfa/ stay until / unless a later perf decision motivates Rust ports.

  • After step 12 (cutover), the upstream C library libast is effectively retired from active development. The submodule pointer may stop being bumped at that point; the oracle tests continue to validate against the historical C reference.

Vendored C dependencies#

The Rust repo owns its own copies of the vendored C sources at vendor/. They are not pulled from the reference submodule at build time, so the everyday workflow does not need it.

pal (vendor/pal/) and erfa (vendor/erfa/) are compiled by ast-core’s build.rs using the cc crate, with the same symbol-renaming conventions palwrap.c / pal2ast.h / erfa2ast.h use today, keeping them private to ast-core. Porting to Rust is a later perf decision, not a phase-one goal — the C source is small, well-tested, and stable.

wcslib (vendor/wcslib/) is treated differently. The fork is ~20 years old and upstream has diverged too far to re-adopt; @timj’s conclusion on the PR was that once we are reimplementing AST we should also take on the wcslib port to remove the FFI crossing and to expose upstream’s newer vectorised transform API to WcsMap. That port is sequenced as step 6a of the indicative delivery order, immediately before WcsMap itself. Until then, vendor/wcslib/ continues to be built as C by ast-core’s build.rs and called through a small private FFI. After step 6a it is removed from the repo entirely.

cminpack is not copied across — it is retired in favour of the levenberg-marquardt crate (already validated in the prototype port at lsst-dm/starlink-astrs). Before retirement, fixture-level validation confirms Rust-LM output matches cminpack output to within fitting tolerances; the cminpack source we validate against lives inside the reference submodule and is not vendored into the new repo.

vendor/README.md records provenance for each copied directory: the upstream URL it came from, the commit hash and date of the import, and a list of any AST-specific local modifications applied. This keeps the chain of custody legible without relying on git history of the source repos.

Object model#

Sibling traits, no hierarchy#

The v1 trait hierarchy (Object → Mapping → Frame → FrameSet) is abandoned. Instead, each public capability is its own trait, at the same level:

pub trait Mapping {
    fn nin(&self) -> usize;
    fn nout(&self) -> usize;
    fn is_inverted(&self) -> bool;
    fn transform(&self, pts: &PointSet, forward: bool) -> AstResult<PointSet>;
    fn transform_into(&self, pts: &PointSet, forward: bool,
                      out: &mut PointSet) -> AstResult<()>;
    fn invert(&mut self);
    fn simplified(&self) -> AstResult<DynMapping>;
    fn rate(&self, at: &[f64], ax1: usize, ax2: usize) -> AstResult<f64>;
    // decompose, map_box, map_split, linear_approx, quad_approx ...
}

pub trait Frame {
    fn naxes(&self) -> usize;
    fn format(&self, axis: usize, value: f64) -> AstResult<String>;
    fn unformat(&self, axis: usize, text: &str) -> AstResult<(f64, usize)>;
    fn distance(&self, a: &[f64], b: &[f64]) -> AstResult<f64>;
    fn angle(&self, a: &[f64], b: &[f64], c: &[f64]) -> AstResult<f64>;
    fn offset(&self, from: &[f64], to: &[f64], dist: f64) -> AstResult<Vec<f64>>;
    fn norm(&self, coords: &mut [f64]) -> AstResult<()>;
    fn convert(&self, to: &dyn Frame, domains: &str) -> AstResult<Option<FrameSet>>;
    // resolve, intersect, axnorm, matchaxes, axdistance, axoffset ...

    // Self-identifying metadata used by FrameSet / Region integrity:
    fn snapshot(&self) -> DynFrame;                    // deep clone for delta computation
    fn map_from(&self, previous: &dyn Frame) -> AstResult<DynMapping>;
}

pub trait Region: Frame + Mapping {
    fn point_in_region(&self, point: &[f64]) -> AstResult<bool>;
    fn overlap(&self, other: &dyn Region) -> AstResult<Overlap>;
    fn negated(&self) -> DynRegion;
    // mask, mesh, bounds ...
}

Notes:

  • trait Mapping and trait Frame are independent — neither extends the other. This matches @TallJimbo / @dsberry’s converged position.

  • trait Region: Frame + Mapping — Region keeps both super-traits, since per @dsberry it must work as both an inside/outside predicate (Mapping) and a coordinate-system descriptor (Frame), and that combination is load-bearing for the region-integrity feature (changing the coordinate system auto-transforms the defining points).

  • Frame::snapshot / Frame::map_from are the primitives used by FrameSet integrity (described later). They’re on Frame because every Frame subclass needs to know how to produce a deep-clone of itself and how to derive the Mapping to a previous version. The default map_from uses Frame::convert under the hood.

  • No public Object trait. Per @TallJimbo’s analysis, the bare Object methods cannot express real cross-class operations (e.g. writing to a Channel is dispatched per-concrete-type, not via Object methods). A private Object super-trait may exist inside ast-core as an implementation detail for shared bookkeeping (class name, ID, Ident), but it is not part of the public API.

Concrete classes#

Each public class is a plain Rust struct holding its state by value:

pub struct ZoomMap    { /* fields */ }   impl Mapping for ZoomMap { .. }
pub struct CmpMap     { /* fields */ }   impl Mapping for CmpMap  { .. }
pub struct PolyMap    { /* fields */ }   impl Mapping for PolyMap { .. }
// ... one struct per public Mapping subclass

pub struct BasicFrame { /* fields */ }   impl Frame for BasicFrame { .. }
pub struct SkyFrame   { /* fields */ }   impl Frame for SkyFrame   { .. }
pub struct SpecFrame  { /* fields */ }   impl Frame for SpecFrame  { .. }
pub struct CmpFrame   { /* fields */ }   impl Frame for CmpFrame   { .. }

pub struct FrameSet   { /* fields */ }
impl Frame   for FrameSet { .. }         // acts as its current Frame
impl Mapping for FrameSet { .. }         // acts as base→current Mapping

pub struct Circle     { /* fields */ }
impl Region  for Circle   { .. }
impl Frame   for Circle   { .. }         // via Region : Frame
impl Mapping for Circle   { .. }         // via Region : Mapping
// ... and so on for Ellipse, Polygon, BoxRegion, CmpRegion, PointList, NullRegion, ...

pub struct FitsChan   { /* fields */ }   // Channel is its own trait (below)

The generic n-axis frame produced by C’s astFrame(naxes, ...) is BasicFrame. Box (AST region) is renamed BoxRegion to avoid colliding with std::boxed::Box.

Trait objects and dispatch#

Where the public API needs polymorphism it returns or accepts trait objects, boxed:

pub type DynMapping = Box<dyn Mapping + Send>;
pub type DynFrame   = Box<dyn Frame   + Send>;
pub type DynRegion  = Box<dyn Region  + Send>;

Concrete-typed call sites (e.g. ZoomMap::new(...).transform(...)) use static dispatch for zero overhead. Heterogeneous collections (a FrameSet’s list of frames; user-composed pipelines) use the Dyn* aliases.

Channel remains a trait (and is borrow-mutable distinct from Mapping / Frame):

pub trait Channel {
    fn read(&mut self) -> AstResult<Option<DynAnyObject>>;   // discriminated by class
    fn write(&mut self, obj: &dyn WritableToChannel) -> AstResult<usize>;
    fn warnings(&self) -> Option<&KeyMap>;
    fn encoding(&self) -> Encoding;
    fn register_intra(&mut self, t: Arc<dyn IntraTransform>);
}

DynAnyObject is an enum of all concrete AST classes that can round-trip through a Channel — enumeration is possible because the set is closed (no extension classes). WritableToChannel is a private trait implemented by every class and used only to dispatch writes; this is the “enumerate concrete types” pattern @TallJimbo flagged as unavoidable.

Ownership, construction, and mutation#

Owned by value; mutate via &mut self#

Every concrete class is held and passed by value. Clone on a concrete class is a deep copy (matching astCopy). Attribute changes mutate in place:

let mut frame = SkyFrame::builder()
    .system(SkySystem::FK5)
    .epoch(2000.0)
    .build()?;

frame.set_epoch(2010.0)?;              // O(1), no allocation, no lock
frame.set_sky_ref_is(SkyRefIs::Origin)?;

let copy = frame.clone();              // deep copy, explicit

This restores the SCUBA-2 hot path: each set_epoch is a single struct field update, same cost as the C version (minus the trailing-underscore variadic overhead).

Builders for construction#

Construction still goes through typestate builders — builders are the one place in the API where an immutable-style chain is idiomatic and the ergonomic payoff is highest:

let frame = SkyFrame::builder()
    .system(SkySystem::FK5)
    .epoch(2000.0)
    .equinox(2000.0)
    .domain("SKY")
    .build()?;

Each concrete class provides builder(). The builder is consumed by build() which returns the owned concrete value (or AstResult<Self> when construction can fail — e.g. invalid attribute combinations).

No edit(), no with_*#

In v1 these existed to work around immutability. They’re gone: direct mutation replaces them. If a user wants a modified copy, they clone() then mutate. For batching, multiple set_* calls on a &mut binding are O(N) field updates, no allocation.

Thread safety#

All concrete classes are Send — they can be moved between threads. None are Sync — they cannot be shared across threads without external synchronisation, because mutation is via &mut self and is deliberately lock-free. This matches the performance constraint from the review: no mutex in the hot paths.

Channels (FitsChan, etc.) are also Send but not Sync — they hold live I/O state.

Channel / Plot split#

  • Value-like, mutable state small: Mapping / Frame / Region concrete classes. Held by value, mutated via &mut self, Clone is deep-copy, Send but not Sync.

  • Stateful I/O: FitsChan, YamlChan, StcsChan, MocChan. Same ownership model; in practice these are not Clone (cloning a half-read channel is nonsensical). Send but not Sync.

  • Stateful drawing: Plot, Plot3D. Held by value, but drawing is routed through a user-supplied &mut dyn Grf passed to each draw call (the Plot itself carries configuration, not the output stream). Plot is Clone (you might want to draw the same Plot over multiple Grf backends).

Error model#

Unchanged from rev 1 except for cosmetics. One canonical error type, one Result alias, no global status in the Rust API.

pub type AstResult<T> = Result<T, AstError>;

#[derive(Debug, Clone)]
pub struct AstError {
    pub code: ErrorCode,
    pub message: String,
    pub context: Vec<String>,
    pub source_location: Option<&'static Location<'static>>,
}

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[non_exhaustive]
pub enum ErrorCode {
    BadAttribute,        // AST__BADAT
    BadBox,              // AST__BADBX
    // ... one variant per AST__XXXX code from ast_err.msg
    Unknown(i32),
}

Rules (unchanged):

  • No global state in ast-core. astOK / astStatus / astWatch etc. exist only in the CFFI shim.

  • thiserror provides boilerplate; anyhow is not used in public APIs.

  • ErrorCode is generated from ast_err.msg by build.rs, keeping Rust and C codes in lockstep.

  • Panics never cross FFI: ast-capi wraps each entry point with catch_unwind and translates panics to AST__INTER.

  • Optional tracing feature emits every constructed AstError as a tracing::error! event; off by default.

Coordinate data#

Two types, matching the C library’s own split between the protected internal PointSet and the public PointList:

use ndarray::{Array2, ArrayView2, ArrayViewMut2};

/// Frame-unaware coordinate block. Used inside Mapping implementations.
/// Row-major: shape = (npoint, ncoord).
pub struct PointSet {
    data: Array2<f64>,
}

impl PointSet {
    pub fn zeros(npoint: usize, ncoord: usize) -> Self;
    pub fn from_rows(rows: impl IntoIterator<Item = impl AsRef<[f64]>>) -> Self;
    pub fn from_columns(cols: &[&[f64]]) -> AstResult<Self>;
    pub fn from_ndarray(a: Array2<f64>) -> Self;

    pub fn npoint(&self) -> usize;
    pub fn ncoord(&self) -> usize;
    pub fn view(&self) -> ArrayView2<f64>;
    pub fn view_mut(&mut self) -> ArrayViewMut2<f64>;
    pub fn into_ndarray(self) -> Array2<f64>;

    pub fn point(&self, i: usize) -> &[f64];
    pub fn point_mut(&mut self, i: usize) -> &mut [f64];
    pub fn axis(&self, j: usize) -> impl Iterator<Item = f64> + '_;
}

/// Frame-aware point collection. Implements Region (inherits from it in
/// the C object model). Changing the Frame triggers Region integrity:
/// the stored positions are transformed through the Frame delta so
/// they continue to describe the same physical locations.
pub struct PointList {
    frame: DynFrame,
    points: PointSet,
}

impl Region for PointList { .. }

  • ndarray is a public dependency of ast-core. The C interop remains cheap: Array2<f64> owns its data as a contiguous row-major Vec<f64> which ast-capi can hand to C without copy. Users who want to avoid the ndarray dependency can still consume .view() / .into_ndarray() outputs trivially.

  • BAD_VALUE = -f64::MAX, bit-identical to AST__BAD (unchanged from v1). NaN accepted on input, BAD_VALUE always on output. Points / PointSet buffers cross the CFFI boundary zero-copy.

  • Transform signature: fn transform(&self, pts: &PointSet, forward: bool) -> AstResult<PointSet>, with transform_into(..., out: &mut PointSet) for buffer reuse.

  • Single-axis / two-axis convenience Mapping::tran1 / tran2 stay (avoid forcing users to allocate a PointSet for 1-D data).

  • Grid transforms fn transform_grid(&self, lbnd: &[i64], ubnd: &[i64], tol: f64, maxpix: usize, forward: bool) -> AstResult<PointSet>.

  • Resample / Rebin generic over SampleType. A DataCube<T> type (sibling of PointSet for N-D gridded data with optional variance) is the input/output. Detailed signatures deferred to the implementation plan.

Generics-over-Frame for compile-time coordinate-system checking was discussed on the PR and dropped: the interaction with AST’s dynamic class structure (needed for serialisation and the C ABI) makes the payoff marginal for this port.

Attribute access#

Typed accessors are the primary surface#

Every concrete class exposes one typed getter, one typed setter, one typed tester, and one typed clearer per public attribute:

impl SkyFrame {
    pub fn system(&self) -> SkySystem;
    pub fn set_system(&mut self, s: SkySystem) -> AstResult<()>;
    pub fn test_system(&self) -> bool;        // attribute explicitly set?
    pub fn clear_system(&mut self) -> AstResult<()>;

    pub fn epoch(&self) -> f64;
    pub fn set_epoch(&mut self, e: f64) -> AstResult<()>;
    pub fn test_epoch(&self) -> bool;
    pub fn clear_epoch(&mut self) -> AstResult<()>;
    // ... one four-tuple per settable attribute
}

Each getter returns the AST-documented default when the attribute is unset, matching C behaviour. test_* distinguishes set from default-used.

Axis-indexed attributes#

A subset of AST attributes are indexed by axis — e.g. Label(1) / Label(2), Unit(axis), Format(axis), SkyRef(1) / SkyRef(2), Symbol(axis), Bottom(axis), Top(axis), Domain(axis) on certain frames, etc. The typed pattern expands naturally:

impl SkyFrame {
    pub fn sky_ref(&self, axis: usize) -> f64;
    pub fn set_sky_ref(&mut self, axis: usize, v: f64) -> AstResult<()>;
    pub fn test_sky_ref(&self, axis: usize) -> bool;
    pub fn clear_sky_ref(&mut self, axis: usize) -> AstResult<()>;
}

impl BasicFrame {
    pub fn label(&self, axis: usize) -> &str;
    pub fn set_label(&mut self, axis: usize, s: impl Into<String>) -> AstResult<()>;
    pub fn test_label(&self, axis: usize) -> bool;
    pub fn clear_label(&mut self, axis: usize) -> AstResult<()>;
    // ... same shape for unit, format, symbol, bottom, top, ...
}

Rules:

  • axis is 1-based in the Rust API to match AST’s documentation, the C API, and the Native serialisation tokens (Lbl1, Lbl2, …). This deliberately violates the usual Rust 0-based convention — the alternative would silently retarget the wrong axis when users translate code or examples from the AST documentation. Documented prominently on every axis-indexed accessor.

  • axis = 0 is rejected with AstError::BadAxis. Out-of-range values (axis > naxes()) produce the same error.

  • Non-indexed equivalents (title, system, epoch) are plain fn title(&self) without the axis argument. The typed accessor set makes the distinction visible at the type level.

What was removed#

  • get_attr(name: &str) -> AttrValue is not in the Rust public API. Users doing programmatic attribute work write a macro or call the typed accessors. Rust’s macro system covers the “loop over attributes” case without runtime string dispatch. The C-binding shim keeps astGetC / astGetI / etc. internally, mapped to the typed getters plus a formatter — users of the C API still see the full astGetC experience.

  • test_attr(name: &str) and clear_attr(name: &str) are not in the Rust public API either. Per @TallJimbo: stringly-keyed attribute presence tests are close kin to Python __getattr__-style tricks and shouldn’t exist without a concrete reason. The typed test_* / clear_* methods cover every attribute the class exposes; the CFFI shim maps astTest / astClear on an attribute-name string to the corresponding typed call (parsing axis indices like SkyRef(1) at the same time).

  • set_fmt("A=1; B=foo") is not in the Rust public API. astSet in C parses the format and dispatches to the typed setters inside the shim.

  • set_many(&[(name, value), ...]) is not in the Rust public API. Use multiple typed setter calls.

  • The edit() builder is not in the Rust public API. Direct mutation replaces it.

Mapping composition#

impl dyn Mapping {
    pub fn then(self: Box<Self>, next: DynMapping) -> AstResult<CmpMap>;
    pub fn alongside(self: Box<Self>, other: DynMapping) -> AstResult<CmpMap>;
}

// Also provided on every concrete Mapping type as inherent methods
// so `zoom.then(wcs)?.then(perm)?` works without up-casting first.
impl ZoomMap {
    pub fn then(self, next: DynMapping) -> AstResult<CmpMap>;
    pub fn alongside(self, other: DynMapping) -> AstResult<CmpMap>;
}
// ... and so on for every concrete Mapping

  • then is series composition (read left-to-right for application order); alongside is parallel. Verbs only — operator sugar (Mul / BitOr) is not provided, per @TallJimbo’s readability concern.

  • Mapping::simplified returns DynMapping; the simplified result may be a different class (UnitMap, a fused MatrixMap, etc.).

  • Mapping::invert takes &mut self and toggles the invert flag in place. (invert() is not inverted() because there is no reason to allocate a new object just to flip a bool.)

  • Compound constructors still accept owned inputs:

    pub fn new(first: DynMapping, second: DynMapping,
           kind: Composition) -> AstResult<CmpMap>;

FrameSet#

Basic API#

pub enum FrameRef<'a> {
    Base,
    Current,
    Index(usize),         // 1-based, matching AST docs and the C API
    Name(&'a str),         // by Ident attribute (see open question below)
}

impl FrameSet {
    pub fn new(base: DynFrame) -> AstResult<FrameSet>;

    pub fn add_frame(&mut self, anchor: FrameRef, map: DynMapping,
                     new_frame: DynFrame) -> AstResult<()>;
    pub fn remove_frame(&mut self, which: FrameRef) -> AstResult<()>;
    pub fn remap_frame(&mut self, which: FrameRef, map: DynMapping) -> AstResult<()>;

    pub fn frame(&self, which: FrameRef) -> AstResult<&dyn Frame>;
    pub fn frame_mut(&mut self, which: FrameRef) -> AstResult<&mut dyn Frame>;
    pub fn mapping(&self, from: FrameRef, to: FrameRef) -> AstResult<DynMapping>;
    pub fn base(&self) -> &dyn Frame;
    pub fn current(&self) -> &dyn Frame;
    pub fn nframe(&self) -> usize;

    pub fn set_base(&mut self, which: FrameRef) -> AstResult<()>;
    pub fn set_current(&mut self, which: FrameRef) -> AstResult<()>;

    pub fn find_frame(&self, template: &dyn Frame, domains: &str)
        -> AstResult<Option<FrameSet>>;
    pub fn add_variant(&mut self, map: DynMapping, name: &str) -> AstResult<()>;
}

impl Frame for FrameSet { .. }       // delegates to current Frame, trapping mutations
impl Mapping for FrameSet { .. }     // base→current mapping

Ownership. DynFrame is Box<dyn Frame + Send> — a single-owner pointer. Calling fs.add_frame(…, new_frame) transfers ownership of new_frame into the FrameSet; the caller can no longer touch that particular Frame instance directly. This is deliberate: if the caller could keep a handle and mutate the Frame behind the FrameSet’s back, the integrity invariant (below) would silently break. Users who want an independent copy of a frame after handing one to a FrameSet clone() it first.

DynMapping follows the same rule — add_frame / remap_frame consume the mapping.

FrameSet integrity#

This is the mechanism that makes attribute mutation on a FrameSet correctly propagate into the internal Mapping chain. Per @dsberry:

Changing a (Frame) attribute on a FrameSet causes the same attribute to be changed within the current Frame. The Mapping from the original Frame to the modified Frame is then found and is used to modify all the Mappings that connect with that Frame within the FrameSet.

Rust implementation:

impl Frame for FrameSet {
    fn set_epoch(&mut self, e: f64) -> AstResult<()> {
        let previous = self.current_frame().snapshot();     // cheap Frame deep-copy
        self.current_frame_mut().set_epoch(e)?;              // direct mutation
        let delta = self.current_frame().map_from(&*previous)?;
        self.fuse_into_current_chain(delta)?;                // integrity update
        Ok(())
    }
    // ... every Frame attribute follows the same pattern
}

The snapshot + map_from primitives on Frame are the reason those methods are part of the trait. Their cost determines how fast per-time-slice FrameSet attribute updates can be; for the common case (a SkyFrame’s Epoch or SkyRefIs changing) snapshot is a small struct copy and map_from returns a precession or rotation Mapping that is trivial to compose. No heap allocation in the common path beyond the resulting CmpMap (which is necessary to record the integrity correction).

Escape hatch: FrameSet::frame_mut(which) returns &mut dyn Frame and lets the caller mutate a Frame without integrity propagation. This matches astFrameSet.getFrame() in C. Documented as the advanced option; typical callers use the delegated setter methods.

Adding a FrameSet to another FrameSet#

Per @dsberry, the semantics depend on the role:

  • fs.add_frame(anchor, map = inner_fs, new_frame = other) — inner_fs is used as a Mapping; it is replaced by its own base→current mapping before insertion. No flattening of inner frames.

  • fs.add_frame(anchor, map, new_frame = inner_fs) — inner_fs is used as a Frame; its frames and mappings are flattened and merged into the outer FrameSet, with the inner’s current Frame being connected to anchor via the supplied map.

The Rust API keeps both by relying on the fact that FrameSet implements both Mapping and Frame. The caller’s choice is visible at the call site. add_frame dispatches on the trait-object types of its arguments.

Ident vs Domain for FrameRef::Name#

@TallJimbo noted astshim used Ident rather than Domain for its FrameDict lookup type. Ident is user-assigned and intended to be unique; Domain groups frames by coordinate family (PIXEL, SKY) and is typically not unique within a FrameSet.

Decision (captured as a commitment): FrameRef::Name looks up by Ident. Rationale: identity of a single frame is the useful semantic for lookup; Domain is for template-matching via find_frame, which already exists. This also matches astshim’s considered choice.

Region#

The Region trait inherits from both Frame and Mapping. Its integrity mechanism mirrors FrameSet’s, per @dsberry:

When you use the Region reference to change the properties of the Frame, it can automatically work out the Mapping from the old Frame to the new Frame and use that Mapping to transform the positions that define the Region.

impl Frame for Circle {
    fn set_system(&mut self, s: SkySystem) -> AstResult<()> {
        let previous = self.frame().snapshot();
        self.frame_mut().set_system(s)?;
        let delta = self.frame().map_from(&*previous)?;
        self.transform_defining_points_in_place(&*delta)?;
        Ok(())
    }
    // ...
}

The same escape-hatch pattern applies: Region::frame_mut() exposes the raw Frame for callers who do not want the defining positions re-projected.

Regions implement Mapping::transform as the inside/outside predicate mapper (point → 0 or 1, or input-passthrough vs BAD_VALUE depending on attributes) used by masking and overlap operations.

Plugin system#

Unchanged from rev 1 in shape; the five traits (Grf, Grf3d, Source, Sink, IntraTransform, Interpolator) still form the extension surface. Error handling is still tracing-based on the Rust side with the astPutErr protocol preserved in the CFFI shim.

Most plugin objects become owned Box<dyn Trait> under the new ownership model (Grf, Grf3d, Source, Sink, Interpolator) — they are handed to a single consumer per use and nothing shares them.

IntraTransform is the exception: it stays Arc<dyn IntraTransform> because a single user transform may be referenced by a long-lived Channel registry and by one or more IntraMap instances read from that channel. Shared ownership is the simplest expression of that lifetime.

Serialisation#

Unchanged from rev 1: the node-tree intermediate, multiple renderers (Native, JSON with schema, XML, YAML, FITS-Native), FITS double formatter with nine-run / zero-run trimming. The to_node / from_node protocol is on a private trait now (since public Object is gone), used internally by the Channel-write dispatch.

// Internal, not in the public prelude:
pub(crate) trait Serializable {
    fn to_node(&self) -> AstResult<Node>;
    fn from_node(n: &Node) -> AstResult<Self> where Self: Sized;
}

The Channel::write(obj: &dyn WritableToChannel) entry enumerates classes via a WritableToChannel private trait with one implementation per concrete class (dispatched by class() -> &'static str). This is the “enumerate the concrete types” pattern @TallJimbo identified as unavoidable.

CFFI shim (ast-capi)#

A single cdylib crate producing libast.{so,dylib,a} with symbols matching the existing ast.h byte-for-byte. Full binary ABI compatibility: existing applications link without recompile.

Handle registry in the owned-value world#

The v1 handle registry held Arc<dyn Object>. Under the new ownership model the shim uses Arc<RefCell<AnyObject>> where AnyObject is the same closed enum used by Channel::read:

enum AnyObject {
    ZoomMap(ZoomMap),
    CmpMap(CmpMap),
    PolyMap(PolyMap),
    // ...
    SkyFrame(SkyFrame),
    FrameSet(FrameSet),
    FitsChan(FitsChan),
    // ...
}

struct Registry {
    slots: RwLock<SlotMap<Handle, Arc<RefCell<AnyObject>>>>,
}

  • astClone = Arc::clone; the same RefCell<AnyObject> is shared across handles, matching C’s ref-count-shared semantics exactly.

  • astAnnul = drop Arc; slot freed at refcount zero. astDelete = force-remove.

  • astSet etc. take &mut on the inner object via RefCell::borrow_mut. No lock (the RefCell is borrow-checked at runtime but not threaded).

  • Thread safety: Arc<RefCell<T>> is Send but not Sync. C callers who want to share a handle across threads must synchronise themselves, exactly matching the current C library’s documented behaviour.

  • The registry’s outer RwLock is acquired only for slot insert/remove/lookup, not for any actual operation on the object.

  • astBegin / astEnd / astExempt are implemented on a per-thread stack of handle sets as before.

The AnyObject enum being closed is acceptable because the AST public class set is closed (no user-extension classes at the class level; users extend through IntraMap, custom Channel sources, custom Grf, etc., all of which the public API already contains). This also matches the Channel-write dispatch requirement.

Global status shim#

Unchanged from rev 1. Thread-local status: i32 plus AstPutErrFun slot. catch_unwind around every extern "C" entry. Translation of Err(AstError) to status + putErr + documented failure sentinel. Source-location fidelity preserved via the C macro capture of __FILE__ / __LINE__.

Variadic astSet and constructors#

Unchanged from rev 1. va_list parsing; format-string tokens dispatched to the typed setters. The absence of set_fmt on the Rust public API does not affect the shim — it implements the parser internally.

Generic resample / rebin#

Each astResampleD / astResampleF / astResampleI / … is its own #[no_mangle] extern "C" fn monomorphising the Rust generic.

Plugin re-exports#

astSetPutErr, astGrfSet / Push / Pop, astIntraReg, astChannel source/sink function pointers all bridge to the Rust traits via the five shims from the plugin section.

ast.h maintenance#

makeh retired. Current ast.h becomes a hand-maintained artifact at crates/ast-capi/include/ast.h. Two CI checks prevent drift (libclang prototype extraction vs nm; cbindgen prototype view vs hand-written header). Macros, error codes, struct typedefs stay by hand. ast_err.h continues to be generated from ast_err.msg by cmake/gen_ast_err_h.cmake, feeding both the C header and the Rust ErrorCode enum.

Test strategy#

Five tiers, each answering a distinct question.

  1. ast-core Rust tests — does the idiomatic API work, and does it cover everything the C suite covered? Lives at crates/ast-core/tests/. The completion goal (a hard project goal — see Goals §4) is full migration of every C test in the upstream ast_tester/ directory:

    • One Rust test file per C test program. testzoommap.c → tests/testzoommap.rs, etc. ~53 files at the time of writing (107 default + 20 PLplot + 1 stress in PLAN.md, with several C programs producing many ctest cases via fixture iteration). Each Rust test has a header comment recording the C ancestor’s path and any idiomatic Rust divergences (same convention used today for Fortran→C ports).

    • All reference fixtures copied verbatim into crates/ast-core/tests/fixtures/:

      • *.head — FITS header inputs for the WCS converter (132 files).

      • *.ast, *.native, *.simp, *.current — Native-encoded AST dumps used by ast_astequal semantic-comparison tests.

      • *.fits-{wcs,iraf,pc,aips,aips++,class,dss}, *.fits-class-roundtrip — FITS-encoding-specific reference outputs from the wcsconverter regression harness.

      • *.attr, *.fattr, *.box — Plot configuration triples used by the plot/grid smoke matrix.

      • *.moctohtml — MOC encoding references.

      • teststc_com, teststc_eg1 … teststc_eg10 — STC-S reference inputs.

      • testxmlchan_com, testmocchan_com — channel-test inputs.

    • The three regression harnesses become Rust binaries that drive their fixture sets:

      • wcsconverter (Rust port of ast_tester/wcsconverter.c): reads .head, writes in each requested encoding, diffs against the matching .fits-* reference. Powers ~14 wcsconv_* ctests + 20 wcsconv_*_native_astequal ctests today.

      • simplify (Rust port of ast_tester/simplify.c): drives .simp / .current references via 3 + 3 ctests today.

      • Plot / grid smoke matrix: ports testplotter.c and testgrid.c to a single Rust harness driven by the .attr / .fattr / .box triples per fixture (40 ctests).

    • All 13 conversion batches in PLAN.md are ported. PLAN.md gains a “C → Rust test migration” section that tracks completion the way it tracks today’s Fortran→C migrations.

    • Tier-1 migration is paced with the implementation, not batched at the end. Each step in the delivery order ports the ast_tester programs whose dependencies it has just made buildable: simple-Mappings step ports testzoommap / testpermmap / testcmpmap / etc.; the BasicFrame step ports testframe / testaxis; the FrameSet step ports testframeset / testconvert; the serialisation step ports testchannel / testxmlchan / the ast_astequal regression harness; and so on. Step 11 of the delivery order is the completion gate that confirms every program and fixture has been ported and is green — not the step at which the bulk of migration work happens. The implementation plan (next document) enumerates the per-step test set; this spec only commits to the end-state.

  2. Native-format regression tests — byte-exact round-trip against captured C output. Fixtures captured from the current C library are committed; Rust reads them, writes them back, diff must be clean. Catches attribute-ordering drift and float-precision changes.

  3. ast-capi C test harness — all CMake tests from reference/starlink-ast/ast_tester/ link against the Rust-built libast and must pass unchanged. Skipped (with a notice) when the submodule is not initialised. This tier proves the CFFI layer preserves the C ABI; tier 1 already proves the Rust API itself covers the same surface.

  4. Oracle correctness tests (ast-oracle-tests) — matched inputs driven through both ast-core and the C libast.a built from the reference/starlink-ast/ submodule. Submodule pinned to a specific commit; bumping the pin is an explicit, reviewable action. Bit-exact for integer/classification outputs; bounded ULP tolerance for floating-point; property-based for random pipelines. Skipped (with a notice) when the submodule is not initialised.

  5. Oracle benchmarks (ast-oracle-bench) — criterion benches of matched workloads on 10k / 100k / 1M point clouds. Rust/C ratio reported; >10% slower fails CI.

New benchmark case added for rev 2: per-time-slice attribute mutation hot path. SkyFrame::set_epoch and SkyFrame::set_sky_ref_is in a tight loop (as in SMURF’s smf_check_coords.c and smf_cubebounds.c) must be within 10% of the C timing. This is the specific performance constraint that drove the v1→v2 redesign.

PLAN.md gains a “C → Rust test migration” section tracking tier-1 progress the same way it tracks today’s Fortran → C migrations.

Indicative delivery order#

  1. Project bootstrap. Initialise the new starlink-astrs repo, copy pal / erfa / wcslib from the current C tree into vendor/ with a vendor/README.md recording provenance, and add reference/starlink-ast/ as a submodule pinned to a known-good commit. CI configured to run tier-1 / tier-2 / tier-5 tests on every PR; tier-3 / tier-4 (submodule-dependent) on a nightly job.

  2. Workspace, error model, traits (Mapping, Frame, Region), both coordinate array types (PointSet, PointList).

  3. Simplest Mappings: UnitMap, ZoomMap, ShiftMap, MatrixMap, PermMap, CmpMap.

  4. BasicFrame, CmpFrame. Run testframe / testcmpframe in Rust.

  5. FrameSet including the integrity mechanism. Run testframeset.

  6. Serialisation: node tree, Native renderer + parser, JSON renderer + schema. Native round-trip regression tests go green.

  7. Harder Mappings: PolyMap (+ levenberg-marquardt), LutMap, MathMap. 6a. Rust port of vendored wcslib, with WcsMap as its first consumer. Delivers the FFI-free hot path for projections and exposes upstream’s vectorised transform API to WcsMap. pal and erfa remain vendored C. Oracle tests validate bit-compatibility against the old C wcslib implementation; once green, vendor/wcslib/ is deleted from the repo and the cc-driven wcslib build is removed from ast-core’s build.rs.

  8. SkyFrame, SpecFrame, TimeFrame, FluxFrame, SpecFluxFrame. Oracle benchmarks for per-time-slice attribute mutation go green here.

  9. Channel, FitsChan (including FITS double formatter), YamlChan, StcsChan, MocChan.

  10. Regions (including region integrity).

  11. Plot / Plot3D + ast-plugins crate (PLplot, SVG, logging backends).

  12. Tier-1 completeness gate. Confirms every C test program in ast_tester/ has a green Rust counterpart and every reference fixture has landed under crates/ast-core/tests/fixtures/. By this point most of the migration has happened incrementally inside steps 2-10 (each step pulling in the tests its classes enable); step 11 audits coverage, ports any stragglers (typically cross-cutting harnesses like testhuge and edge-case-only programs that depend on many subsystems), and confirms the wcsconverter / simplify / plot+grid Rust harnesses match their C ancestors fixture-for-fixture. Concurrently, ast-capi reaches full export coverage and ast-ffi-tests (tier 3) runs the whole C test matrix green against the Rust-built libast. The port is not declarable complete before this step.

  13. Cutover: ast.h now comes from the Rust build; vendored C source for AST itself is retired. pal / erfa remain vendored C.

Known inherited limitations#

The following limitations of the current C library are called out in the A&C paper’s “what we would do differently” section. They cannot be fixed in this port without breaking the C ABI that existing applications link against, so they are accepted as inherited:

  • SkyFrame uses radians, not degrees. The decision pre-dates AST’s unit-conversion machinery, and SkyFrame has been unable to benefit from it because format strings like "hh:mm:ss" conflate unit conversion with formatting. A future Rust-only SkyFrame successor could fix this; the CFFI-compatible SkyFrame cannot.

  • Coordinate system vs domain conflation. The A&C authors note that CoordinateSystem (the mathematical abstraction) and Domain (the physical space) ought to have been different classes. Re-splitting them would require new public APIs; out of scope.

  • Double-precision time axes. Time axes sometimes need precision beyond f64; the public API is f64-only. Not changing here. A future enhancement (proposed by @timj on the PR) could add an optional fixed-offset MJD attribute to TimeFrame, defaulting to zero, so that internal transforms can use ERFA’s two-double jd1 + jd2 API for full precision while keeping the public surface f64. That requires a new public attribute, so it’s a forward enhancement rather than a fix to the existing port.

Open questions deferred to the implementation plan#

  • Second, cleaner C ABI (ast2.h). Not on day one; revisit after the port is complete.

  • DataCube<T> detailed signature. Largest deferred piece.

  • Per-class builder and accessor enumeration for all ~50 Mapping subclasses, all Frame subclasses, all Region subclasses. Mechanical once the patterns are implemented for representative cases.