terminal - How to fix column calculation in Python readline if using color prompt

Question

I use standard tips for customizing interactive Python session:

$ cat ~/.bashrc
export PYTHONSTARTUP=~/.pystartup

$ cat ~/.pystartup
import os
import sys
import atexit
import readline
import rlcompleter

historyPath = os.path.expanduser("~/.pyhistory")

def save_history(historyPath=historyPath):
    import readline
    readline.write_history_file(historyPath)

if os.path.exists(historyPath):
    readline.read_history_file(historyPath)

term_with_colors = ['xterm', 'xterm-color', 'xterm-256color', 'linux', 'screen', 'screen-256color', 'screen-bce']
if os.environ.get('TERM') in term_with_colors:
    green='33[32m'
    red='33[31m'
    reset='33[0m'
    sys.ps1 = red + '>>> ' + reset
    sys.ps2 = green + '... ' + reset
del term_with_colors

atexit.register(save_history)
del os, sys, atexit, readline, rlcompleter, save_history, historyPath

Now I get context sensitive completion and color prompt.

The problem comes from color prompt - when I invoke history-search-backward (by pressing UP) in an interactive Python session, Readline takes in account terminal escape sequences, so the cursor position is wrongly calculated and the text is wrongly displayed.

In Bash man page this problem is mentioned and fixed by special markers:

    [     begin a sequence of non-printing characters,
           which could be used to embed a
           terminal control sequence into the prompt
    ]     end a sequence of non-printing characters

How to fix this issue for Python prompt?

Answer 1

I open info readline and found:

 -- Function: int rl_expand_prompt (char *prompt)
     Expand any special character sequences in PROMPT and set up the
     local Readline prompt redisplay variables.  This function is
     called by `readline()'.  It may also be called to expand the
     primary prompt if the `rl_on_new_line_with_prompt()' function or
     `rl_already_prompted' variable is used.  It returns the number of
     visible characters on the last line of the (possibly multi-line)
     prompt.  Applications may indicate that the prompt contains
     characters that take up no physical screen space when displayed by
     bracketing a sequence of such characters with the special markers
     `RL_PROMPT_START_IGNORE' and `RL_PROMPT_END_IGNORE' (declared in
     `readline.h'.  This may be used to embed terminal-specific escape
     sequences in prompts.

As text suggested I searched for RL_PROMPT_START_IGNORE and RL_PROMPT_END_IGNORE definition in readline.h and found next:

/* Definitions available for use by readline clients. */
#define RL_PROMPT_START_IGNORE  '01'
#define RL_PROMPT_END_IGNORE    '02'

So I put appropriate changes to my ~/.pystartup:

    green='0133[32m02'
    red='0133[31m02'
    reset='0133[0m02'

and now all work fine!!!