irb on Windows, ironruby starts irb, no CRuby gems

Docs/releases/v4.0.3.md

@@ -31,6 +31,23 @@ gh attestation verify ironruby-4.0.3-win-x64-setup.exe --repo Largo/ironruby
   Setup 6.7.3 from `Util/installer/ironruby.iss`, and installed, run and uninstalled there
   before it is published.
 
+### Changed
+
+- **`ironruby` at a terminal starts irb**, as `python` starts its interactive interpreter:
+  with no script and no `-e`, and stdin a terminal, `ironruby` is `iirb`. From a pipe or a
+  file it still reads the program from stdin, as `ruby` does, and so do `ironruby -`,
+  `-n`, `-p`, `-c` and `-x`.
+- **IronRuby no longer uses a CRuby's gems.** It used to offer the gem directory of a CRuby
+  4.0 found on the PATH for third-party gems. That loaded CRuby's code into IronRuby: with
+  CRuby on the PATH, `igem install` loaded CRuby's rdoc plugin over the rdoc IronRuby ships
+  ("already initialized constant RDoc::VERSION") and then failed generating the gem's
+  documentation. IronRuby now reads only its own gem directories; a gem it needs is
+  installed for it, with `igem install` or Bundler. `IRONRUBY_NO_HOST_GEMS` is gone with it.
+- **`igem install` builds no documentation unless asked**, as on TruffleRuby: generating ri
+  and rdoc takes longer than installing most gems. `igem install --document` (or
+  `gem: --document` in `~/.gemrc`) builds it with the rdoc IronRuby ships, and uninstalling
+  the gem removes it again.
+
 ### Fixes
 
 - **irb starts on Windows again.** In 4.0.2 it stopped at once with `unknown symbol
@@ -40,3 +57,10 @@ gh attestation verify ironruby-4.0.3-win-x64-setup.exe --repo Largo/ironruby
   IronRuby that was an index, not an address. It is a real address now, of a block holding
   the String's bytes, and what C writes there through Fiddle comes back into the String, as
   with CRuby - so reline runs as Ruby ships it.
+- **irb shows results on Windows.** After the first result it stopped with
+  `Errno::ENOTTY` from `IO#winsize`: IronRuby's io/console asked only termios for the
+  terminal size, which Windows does not have. It now asks the console, as CRuby does, and
+  raises `Errno::EBADF` off a console, as CRuby does on Windows.
+- **`require 'ripper'` gives `Ripper::Filter`**, as in CRuby. rdoc's Ruby parser subclasses
+  it, and generating a gem's documentation failed with `uninitialized constant
+  Ripper::Filter`.

README.md

@@ -280,12 +280,11 @@ versions CRuby 4.0.6 bundles, so `require "rexml/document"` and `ir -S rake` wor
 every machine, with or without a CRuby next to IronRuby.
 [`Util/install-bundled-gems.rb`](Util/install-bundled-gems.rb) installs them from a CRuby
 source tarball's `gems/*.gem` and lists the ones left out and why (the C extensions IronRuby
-implements itself, and the ones that need `rbs`).  A host CRuby 4.0's gem directory is still
-offered for third-party gems, but never for a gem IronRuby ships;
-`IRONRUBY_NO_HOST_GEMS=1` turns it off entirely.
+implements itself, and the ones that need `rbs`).  IronRuby never uses the gems of a CRuby
+installed on the same machine: a gem it needs is installed for IronRuby, with `igem install`
+or Bundler.
 
-A C extension is never built: there is no `ruby.h` to compile against, and the gem whose
-extension is missing is often in a read-only host CRuby tree.  Such a gem is left alone -
+A C extension is never built: there is no `ruby.h` to compile against.  Such a gem is left alone -
 its `.rb` files load, a real `require` of the `.so` raises `LoadError`, and the pure-Ruby
 fallback most of them carry is used instead.
 

Src/IronRuby.Tests/RubyTests.cs

@@ -733,6 +733,8 @@ public Tests(Driver/*!*/ driver) {
                 Fiddle_AnsiSuffix1,
                 PackPointer_RealAddress1,
                 PackPointer_RelineWin32Api1,
+                IoConsole_WinsizeOffTerminal1,
+                Ripper_Filter1,
                 Dir_ChdirRelative1,
                 File_ExpandTildeOrder1,
                 File_FlockReadpartialRlimit1,

Src/IronRuby.Tests/Runtime/MiscTests.cs

@@ -1093,6 +1093,57 @@ p File.expand_path('~') == (windows ? 'Y:/profile' : File.expand_path('~'))
 ");
         }
 
+        /// <summary>
+        /// IO#winsize off a terminal raises what MRI raises: ENOTTY from ioctl on Linux and macOS,
+        /// EBADF from the console API on Windows, where IronRuby used to have no winsize at all -
+        /// irb asks $stdout for it before showing each result, and died after the first one.
+        /// A console's own size needs a console, which a test run does not have.
+        /// </summary>
+        public void IoConsole_WinsizeOffTerminal1() {
+            TestOutput(@"
+require 'ruby4'
+require 'io/console'
+require 'tmpdir'
+windows = File::ALT_SEPARATOR == '\\'
+path = File.join(Dir.tmpdir, ""ir-winsize-#{Process.pid}.txt"")
+File.open(path, 'w') do |f|
+  begin
+    f.winsize
+  rescue SystemCallError => e
+    p e.class == (windows ? Errno::EBADF : Errno::ENOTTY)
+  end
+end
+File.delete(path)
+", @"
+true
+");
+        }
+
+        /// <summary>
+        /// require 'ripper' gives Ripper::Filter, as MRI's ripper.rb does: rdoc's Ruby parser
+        /// subclasses it, and `gem install` generating documentation died on the NameError.
+        /// </summary>
+        public void Ripper_Filter1() {
+            TestOutput(@"
+require 'ruby4'
+require 'ripper'
+class Words < Ripper::Filter
+  def on_default(event, tok, data)
+    data << [lineno, column, event, tok]
+  end
+end
+Words.new(""x = 1 # one\n"").parse([]).each { |t| p t }
+", @"
+[1, 0, :on_ident, ""x""]
+[1, 1, :on_sp, "" ""]
+[1, 2, :on_op, ""=""]
+[1, 3, :on_sp, "" ""]
+[1, 4, :on_int, ""1""]
+[1, 5, :on_sp, "" ""]
+[1, 6, :on_comment, ""# one\n""]
+");
+        }
+
         /// <summary>
         /// What RubyGems and Bundler need from the file system on Windows, where each of these
         /// used to P/Invoke libc or raise: File#flock (RubyGems locks every binstub it writes),

Src/Libraries/Termios/Termios.cs

@@ -15,6 +15,7 @@
 using IronRuby.Builtins;
 using IronRuby.Runtime;
 using Microsoft.Scripting.Runtime;
+using Microsoft.Scripting.Utils;
 
 namespace IronRuby.StandardLibrary.Termios {
 
@@ -31,7 +32,8 @@ namespace IronRuby.StandardLibrary.Termios {
     ///
     /// The constants are Linux's (asm-generic/termbits.h, asm-generic/ioctls.h).  On a
     /// platform without termios <see cref="Supported"/> is false and every call returns nil
-    /// or false, which io/console.rb turns into the ENOTTY MRI raises off a non-terminal.
+    /// or false, which io/console.rb turns into the error MRI raises off a non-terminal -
+    /// except winsize, which Windows answers from the console API, as MRI does there.
     /// </summary>
     [RubyModule("Termios")]
     public static class TermiosOps {
@@ -181,10 +183,15 @@ public static bool TcFlush(RubyModule/*!*/ self, [NotNull]RubyIO/*!*/ io, [Defau
         #region winsize
 
         /// <summary>
-        /// ioctl(TIOCGWINSZ) as [rows, columns], or nil when this is not a terminal.
+        /// ioctl(TIOCGWINSZ) as [rows, columns], or nil when this is not a terminal.  On Windows
+        /// it is the console window's size, from GetConsoleScreenBufferInfo, as MRI's
+        /// io/console answers it there - irb asks $stdout for it before it shows each result.
         /// </summary>
         [RubyMethod("winsize", RubyMethodAttributes.PublicSingleton)]
         public static object WinSize(RubyModule/*!*/ self, [NotNull]RubyIO/*!*/ io) {
+            if (!_supported) {
+                return WindowsWinSize(io);
+            }
             int fd = DescriptorOf(io);
             if (fd < 0) {
                 return null;
@@ -218,6 +225,52 @@ public static bool SetWinSize(RubyModule/*!*/ self, [NotNull]RubyIO/*!*/ io,
             return sys_ioctl(fd, TIOCSWINSZ, buffer) == 0;
         }
 
+        private const int STD_INPUT_HANDLE = -10;
+        private const int STD_OUTPUT_HANDLE = -11;
+        private const int STD_ERROR_HANDLE = -12;
+
+        /// <summary>
+        /// CONSOLE_SCREEN_BUFFER_INFO: dwSize, dwCursorPosition (COORDs), wAttributes, then
+        /// srWindow - Left, Top, Right, Bottom as shorts at 10, 12, 14 and 16 - and
+        /// dwMaximumWindowSize.  22 bytes, laid out by hand like struct termios above.
+        /// </summary>
+        private const int ScreenBufferInfoSize = 22;
+
+        [DllImport("kernel32", SetLastError = true)]
+        private static extern IntPtr GetStdHandle(int nStdHandle);
+
+        [DllImport("kernel32", SetLastError = true)]
+        private static extern bool GetConsoleScreenBufferInfo(IntPtr hConsoleOutput, byte[] info);
+
+        /// <summary>
+        /// The visible window of the console a standard stream is attached to, as MRI computes
+        /// it: srWindow.Bottom - Top + 1 rows and Right - Left + 1 columns.  nil for an IO
+        /// that is no standard stream, and for one that is not a console - redirected, or
+        /// the input side, which has no screen buffer.
+        /// </summary>
+        private static object WindowsWinSize(RubyIO/*!*/ io) {
+            if (io.Closed) {
+                return null;
+            }
+            int std;
+            switch (io.ConsoleStreamType) {
+                case ConsoleStreamType.Input: std = STD_INPUT_HANDLE; break;
+                case ConsoleStreamType.Output: std = STD_OUTPUT_HANDLE; break;
+                case ConsoleStreamType.ErrorOutput: std = STD_ERROR_HANDLE; break;
+                default: return null;
+            }
+            byte[] info = new byte[ScreenBufferInfoSize];
+            if (!GetConsoleScreenBufferInfo(GetStdHandle(std), info)) {
+                return null;
+            }
+            short left = BitConverter.ToInt16(info, 10), top = BitConverter.ToInt16(info, 12);
+            short right = BitConverter.ToInt16(info, 14), bottom = BitConverter.ToInt16(info, 16);
+            var result = new RubyArray(2);
+            result.Add(bottom - top + 1);
+            result.Add(right - left + 1);
+            return result;
+        }
+
         #endregion
 
         #region constants

Src/Ruby/Builtins/RubyIO.cs

@@ -218,8 +218,15 @@ public ConsoleStreamType? ConsoleStreamType {
                 if (Closed) {
                     return null;
                 }
-                var stream = GetStream();
-                var console = stream.BaseStream as ConsoleStream;
+                // A copy that could not dup(2) the descriptor - #dup of a console stream on
+                // Windows - wraps the original's buffered stream in one of its own, so the
+                // console may be more than one buffer down.
+                Stream stream = GetStream().BaseStream;
+                RubyBufferedStream buffered;
+                while ((buffered = stream as RubyBufferedStream) != null) {
+                    stream = buffered.BaseStream;
+                }
+                var console = stream as ConsoleStream;
                 return console != null ? console.StreamType : (ConsoleStreamType?)null;
             } 
         }

Src/Ruby/Hosting/RubyCommandLine.cs

@@ -84,9 +84,9 @@ protected override int Run() {
 
             // A script argument of "-" says to read the program from stdin, and is how the CRuby
             // test suite runs a program in a child process; ARGV is what follows it. The options
-            // parser gives "-" when there is neither -e nor a script, so `ironruby` alone reads
-            // stdin as `ruby` does - at a terminal too, until EOF. A host that skips the parser
-            // and gives neither still gets stdin when it is redirected.
+            // parser gives "-" when there is neither -e nor a script and stdin is not a terminal
+            // (at one it gives irb), so `ironruby` alone reads a piped program as `ruby` does. A
+            // host that skips the parser and gives neither still gets stdin when it is redirected.
             if (Options.Command == null &&
                 (Options.FileName == "-" || (Options.FileName == null && System.Console.IsInputRedirected))) {
                 // The program is read as bytes: a magic comment may say they are not UTF-8, and

Src/Ruby/Hosting/RubyOptionsParser.cs

@@ -843,13 +843,40 @@ protected override void AfterParse() {
                 ConsoleOptions.Exit = true;
             }
 
-            // `ruby` with neither -e nor a script reads the program from stdin until EOF - at a
-            // terminal too - as if "-" had been given, and $0 is "-" (MRI, JRuby). IronRuby used to
-            // start the DLR's interactive console there; irb is `irb` / `iirb`. ironrubyw, which has
-            // no console, then reads nothing and exits, as rubyw does.
+            // `ruby` with neither -e nor a script reads the program from stdin until EOF, as if "-"
+            // had been given, and $0 is "-" (MRI, JRuby); so does `ironruby` when stdin is a pipe
+            // or a file. At a terminal it starts irb instead, as `python` starts its interactive
+            // interpreter there: `ironruby` alone is then `iirb`, the -S irb that ironruby.cmd and
+            // ironruby find in RUBYPATH. An explicit "-", and -n, -p, -c and -x, which are about
+            // stdin, still read it. ironrubyw, which has no console, reads nothing and exits, as
+            // rubyw does. Only an irb found in RUBYPATH or PATH runs: FindMainFileFromPath hands
+            // back a name it did not find unchanged, and that would be a file called irb in the
+            // current directory.
             if (!ConsoleOptions.Exit && ConsoleOptions.Command == null && ConsoleOptions.FileName == null) {
-                ConsoleOptions.FileName = "-";
-                LanguageSetup.Options["MainFile"] = "-";
+                string irb = StartsInteractive() ? FindMainFileFromPath("irb") : null;
+                if (irb != null && irb != "irb" && Platform.FileExists(irb)) {
+                    ConsoleOptions.FileName = irb;
+                    SetupOptionsForMainFile();
+                } else {
+                    ConsoleOptions.FileName = "-";
+                    LanguageSetup.Options["MainFile"] = "-";
+                }
+            }
+        }
+
+        /// <summary>
+        /// Whether `ironruby` with no program should be interactive: stdin is a terminal (a
+        /// console on Windows, which .NET tells apart from NUL), and no option asked for stdin.
+        /// </summary>
+        private bool StartsInteractive() {
+            if (LanguageSetup.Options.ContainsKey("LoopOverInput") || LanguageSetup.Options.ContainsKey("CheckSyntaxOnly") ||
+                LanguageSetup.Options.ContainsKey("SkipToRubyShebang")) {
+                return false;
+            }
+            try {
+                return !Console.IsInputRedirected;
+            } catch (IOException) {
+                return false;
             }
         }
 

Src/StdLib/ironruby/io/console.rb

@@ -203,10 +203,12 @@ def ttymode
 
   # -------------------------------------------------------------------- size
 
-  # [rows, columns] of the terminal this IO is connected to.
+  # [rows, columns] of the terminal this IO is connected to.  Off one, MRI raises
+  # ENOTTY from ioctl, and on Windows EBADF - the error GetConsoleScreenBufferInfo
+  # leaves for a handle that is no console output, STDIN's included.
   def winsize
     size = ::Termios.winsize(self)
-    raise Errno::ENOTTY, __console_name__ unless size
+    raise (::Termios.supported? ? Errno::ENOTTY : Errno::EBADF), __console_name__ unless size
     size
   end
 

Src/StdLib/ironruby/ripper.rb

@@ -704,3 +704,7 @@ def match(n = 0)
 end
 
 require 'ripper/sexp_builder'
+# MRI's ripper.rb requires ripper/filter too; rdoc's Ruby parser subclasses Ripper::Filter.
+# An autoload, as filter.rb requires ripper/lexer, which here is this file: required from
+# here it would be a circular require.
+Ripper.autoload :Filter, 'ripper/filter'