x/crypto/ssh: Session.Run() returns error EOF

[Farioko]

Hi,

I'm trying to run commands on a router through SSH. The program works fine if I run it against an OpenSSH server, but against the router with its own proprietary implementation of ssh, Session.Run() returns the error EOF. I'm running Go 1.6.2 and tried executing this program on Linux amd64, Windows amd64, Windows i386. Is this a bug and is there a work around?

package main

import(
        "log"
        "bytes"
        "golang.org/x/crypto/ssh"
)

func main() {
        config := &ssh.ClientConfig{
                User: "admin",
                Auth: []ssh.AuthMethod{ssh.Password("admin")},
                Config: ssh.Config{
                        Ciphers: []string{"aes128-cbc"},
                },
        }

        client, err := ssh.Dial("tcp", "192.168.1.1:22", config)
        if err != nil {
                log.Fatalln("Unable to connect: " + err.Error())
        }

        defer client.Close()

        session, err := client.NewSession()
        if err != nil {
                log.Fatalln("Unable to open session: " + err.Error())
        }

        var b bytes.Buffer
        session.Stdout = &b

        err = session.Run("show Device")
        if err != nil {
                log.Fatalln("Unable to run command: " + err.Error())
        }

        log.Println(b.String())
}

The program returns the following:

Unable to run command: EOF.

[ianlancetaylor]

CC @agl @hanwen

[hanwen]

There is a debugMux variable that you could set which will print what goes over the wire. That might explain in more detail what is going on. WIthout further info, I'd assume the remote device is picky about how the command is formatted, and closes the connection on any problem.

[Farioko]

@hanwen Thank you for answering. I do know that whenever I exit, the server returns status code -1. Could that be causing the problem?

2016/06/28 19:44:50 send global(1): ssh.channelOpenMsg{ChanType:"session", PeersId:0x1, PeersWindow:0x200000, MaxPacketSize:0x8000, TypeSpecificData:[]uint8(nil)}
2016/06/28 19:44:50 decoding(1): 91 &ssh.channelOpenConfirmMsg{PeersId:0x1, MyId:0x11c0b511, MyWindow:0x0, MaxPacketSize:0x88b8, TypeSpecificData:[]uint8{}} - 17 bytes
2016/06/28 19:44:50 send(1): ssh.channelRequestMsg{PeersId:0x11c0b511, Request:"exec", WantReply:true, RequestSpecificData:[]uint8{0x0, 0x0, 0x0, 0xb, 0x73, 0x68, 0x6f, 0x77, 0x20, 0x44, 0x65, 0x76, 0x69, 0x63, 0x65}}
2016/06/28 19:44:50 decoding(1): data packet - 11 bytes
2016/06/28 19:44:50 decoding(1): data packet - 11 bytes
2016/06/28 19:44:50 decoding(1): data packet - 923 bytes
2016/06/28 19:44:50 decoding(1): data packet - 11 bytes
2016/06/28 19:44:50 decoding(1): 97 &ssh.channelCloseMsg{PeersId:0x1} - 5 bytes
2016/06/28 19:44:50 send(1): ssh.channelCloseMsg{PeersId:0x11c0b511}
2016/06/28 19:44:50 decoding(1): 93 &ssh.windowAdjustMsg{PeersId:0x1, AdditionalBytes:0x2800} - 9 bytes
2016/06/28 19:44:50 Unable to run command: EOF

[hanwen]

Our library is waiting for the exit-status or exit-signal message, which isn't coming. According to RFC 4254, those are optional, so we should probably do something intelligent in case the remote end doesn't send a status message.

[Farioko]

I'm happy you found the possible solution, do you know how long it will take before this gets fixed?

[hanwen]

I'd need to set aside some time, which I am short of. I may be able to get to it next week.

If you like challenges, you could try to setup a server inside the unittests that mimicks this failure mode. That would make it easier to see how to fix it.

(I think we should return a special error that signals that the exit code wasn't returned).

[Farioko]

You can actually download the real OS and run it in VMWare/KVM.
If you register an (free) account on clavister.com, then you can download the image of cOS Core.
It will go into lockdown mode after 2 hours, but then you can simply reboot it.

[hanwen]

I mean: we'll need a unittest written in Go that demonstrates the problematic behavior before we attempt a fix. The debug log is enough information to write such a test.

[hanwen]

I have made a CL that handles lack of exit-status more specifically, see https://go-review.googlesource.com/#/c/24727/1

However, this isn't the real problem (turns out we had a test for this all along.) The real problem is here:

2016/06/28 19:44:50 decoding(1): 97 &ssh.channelCloseMsg{PeersId:0x1} - 5 bytes
2016/06/28 19:44:50 send(1): ssh.channelCloseMsg{PeersId:0x11c0b511}
2016/06/28 19:44:50 decoding(1): 93 &ssh.windowAdjustMsg{PeersId:0x1, AdditionalBytes:0x2800} - 9 bytes

after receiving the channel close message, we send a close message too. At this point, channel ID 0x01 is unused. However, the remote side then sends windowAdjustMsg erroneously, to a non-existent channel 0x01, which I would expect to trigger this error path

https://github.com/golang/crypto/blob/master/ssh/mux.go#L243

I don't understand why you see EOF though. If you could insert some debug prints and see which error path triggers it, that would be helpful.

[Farioko]

I'm not sure how to debug this? How do I find the function that returns this error?

[hanwen]

see

https://play.golang.org/p/GBpKnBRL2N

you'd change all error returns to use withLN.

[Farioko]

Your solution didn't work, but I used grep to find any line containing EOF in the library and found where the EOF error gets returned.

It's in channel.go at line 591.

[hanwen]

oh. your device's SSH is screwed up.

2016/06/28 19:44:50 send(1): ssh.channelRequestMsg{PeersId:0x11c0b511, Request:"exec", WantReply:true, RequestSpecificData:[]uint8{0x0, 0x0, 0x0, 0xb, 0x73, 0x68, 0x6f, 0x77, 0x20, 0x44, 0x65, 0x76, 0x69, 0x63, 0x65}}

it's asking for a reply to the "exec" message, which never comes.

I think your best bet is to reimplement 'exec' support from session.go for this device, but changing WantReply to false.