After viewing a page with Big5 charset and next a page with UTF-8
charset iconv was used, so slows down a bit.
Now iconv is used only for those charsets, which has the iconv
bitfield set.
Implement new heartbeat code to catch runaway execution of document
ECMAScript code. The old code uses JS_SetBranchCallback which is
deprecated in new versions of SpiderMonkey. The new code uses setitimer(2)
and the JS_SetOperationCallback and JS_TriggerOperationCallback interfaces,
introduced in SpiderMonkey 1.8.1. Compatibility with both the old
JS_SetBranchCallback and the new interfaces is maintained.
Bring the code closer to ELinks 0.13.GIT commit
71ccbe0f8d made on 2007-11-07.
Don't define a separate download_flags_T typedef though,
because Doxygen generates nicer links if the enum is used directly.
There are two identical tp_show_header() functions: one in
src/session/download.c and one in src/protocol/bittorrent/dialogs.c.
Neither is declared in any header, but the latter was not static.
If msg_box() runs out of memory, it returns NULL. In this case,
the done_handler_T callbacks of the buttons will not be called. So
lookup_unique_name() must instead free the struct lun_hop on its own.
Remove the stracpy(ofile) call that could never be executed.
This removes the need to handle errors from that call,
and makes it clear that lun_hop->file need not be separately freed
if an error occurs in lookup_unique_name().
With GCC 4.3.1 on i686, this changes the sizes of sections as follows:
section before after change
.text 682428 682492 +64
.rodata 212668 216352 +3684
.data 58092 54444 -3648
.debug_info 1482388 1482472 +84
.debug_abbrev 153714 153723 +9
.debug_line 272299 272319 +20
.debug_loc 540394 540372 -22
.debug_ranges 113784 113792 +8
Total 3917695 3917894 +199
The surprising .text change comes from src/config/dialogs.o.
Some of that is in get_keybinding_text(), where GCC changes the
order of basic blocks and apparently misses some optimizations.
I added this bug last night. continue_download_do() passed the
file descriptor to transform_codw_to_cmdw(), which saved it, but
continue_download_do() then closed it.
In common_download(), move session.download_uri to the new member
cmdw_hop.download_uri, which common_download_do() then reads. This
shields the download request against possible session.download_uri
changes made for other downloads. And transform_codw_to_cmdw() no
longer needs to touch session.download_uri at all, solving a FIXME.
Commit ff136e5116 on 2006-07-16 made
lun_resume() check if cdf_hop->data points to struct codw_hop,
and transform that to struct cmdw_hop if so. Move the transform
into a separate function called from continue_download_do().
This way, the structures used with create_download_file() no longer
need to begin with int magic.
To reproduce:
- Configure with --enable-debug.
- Go to http://elinks.cz/
- Set the cursor on the "About" link and press d to download,
- ELinks asks where to save the file. Cancel that with Esc.
This leaves session.download_uri != NULL.
- Go to /etc/passwd
- ELinks asks what to do with the file. Choose to download.
- ELinks asks where to save the file. Type the name of a new file
and press Enter.
- Again go to /etc/passwd
- ELinks asks what to do with the file. Choose to download.
- ELinks asks where to save the file. Type the same name as before
and press Enter.
- ELinks asks whether to resume or overwrite. Choose to resume.
This changes session.download_uri and leaks the original URI.
- Quit ELinks. It reports memory leaks:
0x88936d8:28 @ alloc'd at /home/Kalle/src/elinks-0.12/src/util/hash.c:89
0x88dac00:95 @ alloc'd at /home/Kalle/src/elinks-0.12/src/protocol/uri.c:1551
0x88c33a8:4104 @ alloc'd at /home/Kalle/src/elinks-0.12/src/util/hash.c:41
This commit fixes the leak, but it's still a bug that lun_resume() can
replace the session.download_uri that will be used by another pending
download. In particular, this might happen if the user first presses
d to download, and then while ELinks is asking for the file name, a
web script changes window.location to a different URI and that causes
ELinks to ask what to do with the file. So I'm leaving the FIXME
comment in for now.
If init_file_download() succeeds (returning non-NULL), it saves the
file descriptor to file_download->handle, whence abort_download() will
close it. However, if init_file_download() fails, the caller is
responsible of closing the file, something common_download_do() and
continue_download_do() failed to do. There was no problem with
bittorrent_download() because that uses -1 as the fd.
If init_file_download() succeeds (returning non-NULL), it saves the
file-name pointer to file_download->file, whence abort_download() will
free it. However, if init_file_download() fails, the caller is
responsible of freeing the name. bittorrent_download() already did so
but common_download_do() and continue_download_do() didn't.
Remove enum main_action_offset, enum edit_action_offset, and enum
menu_action_offset. It seems the original plan (in commit
174eabf1a448d3f084a318aab77805828f35c42e on 2005-05-16) was to include
the action flags in the action IDs, perhaps with something like:
ACT_##map##_##action = ACT_##map##_OFFSET_##action | flags
However, this OR operation was never implemented; each ACT_*_*
constant had the same value as the corresponding ACT_*_OFFSET_*,
and the code that looked for flags in action IDs found only zeroes.
Then on 2005-06-10, a separate action.flags member was added, and
the flag checks were corrected to read that instead. So, it seems
safe to say that the original plan has been discarded and the offset
enumerations won't be needed.
Doxygen isn't too good at documenting the parameters of a callback
within the documentation of a parameter that points to the callback.
A typedef provides a better place to document the parameters.
If the user chose File -> Save formatted document and typed the name
of an existing file, ELinks offered to resume downloading the file.
There are a few problems with that:
* save_formatted_finish does not actually support resuming. It would
instead overwrite the beginning of the file and not truncate it.
* When save_formatted calls create_download_file, cdf_hop->data
ends up pointing to struct document. If the user then chooses to
resume, lun_resume would read *(int *)cdf_hop->data, hoping to
get cmdw_hop.magic or codw_hop.magic. struct document does not
begin with any such magic value.
* Because ELinks already has the formatted document in memory,
resuming saves neither time nor I/O.
So don't show the "Resume download of the original file" button in
this situation.
After the recent ecmascript_get_interpreter change, I got an assertion
failure in render_document, which calls ecmascript_reset_state and
then asserts that it has set vs->ecmascript != NULL.
ecmascript_reset_state cannot guarantee that because there might not
even be enough free memory for mem_calloc(1, sizeof(struct
ecmascript_interpreter). So, replace the assertion in render_document
with error handling, and likewise in call_onsubmit_and_submit.
This should fix a crash in:
at /home/Kalle/src/elinks-0.12/src/ecmascript/spidermonkey.c:251
at /home/Kalle/src/elinks-0.12/src/ecmascript/ecmascript.c:104
at /home/Kalle/src/elinks-0.12/src/viewer/text/vs.c:64
It seems that spidermonkey_get_interpreter failed and returned NULL to
ecmascript_get_interpreter, which did not check the return value and
behaved as if the ECMAScript interpreter had been properly initialized.
This caused destroy_vs to call ecmascript_put_interpreter, but
backend_data which should have been a JSContext * was NULL, causing
a crash in SpiderMonkey.
An alternative fix might be to make spidermonkey_put_interpreter skip
the JS_DestroyContext call if ctx is NULL. However, I think it is
better to make sure ecmascript_get_interpreter returns NULL if
spidermonkey_get_interpreter fails, so that vs->ecmascript is left
NULL and there's no chance that some other code might try to
dereference the (JSContext *) NULL.
Perhaps because of bug 981, if one opened hundreds of pages with
elinks --remote openURL(...), then ELinks 0.11.4 could crash with a
SIGSEGV in JS_InitClass called from spidermonkey_get_interpreter.
SpiderMonkey ran out of memory and began returning NULL and JS_FALSE
but ELinks didn't notice them and pressed on. Add some checks to
avoid the crash, although the underlying out-of-memory error remains.
The old code failed to write pending spaces before changing the
background color. That seems hard to fix without duplicating code,
and ELinks pads dumped lines to the requested width in these color
modes anyway, so this commit just makes ELinks write all spaces
immediately when colors are being used.
Try the following command before and after this commit:
elinks --no-home --eval "set document.colors.use_document_colors = 2" \
--dump-color-mode 1 --dump test/color.html
In DUMP_FUNCTION_SPECIALIZED, use isscreensafe_ucs (for UTF-8) or
isscreensafe (for unibyte) to detect control characters, and replace
them with spaces. add_document_to_string already did the same.
In DUMP_FUNCTION_SPECIALIZED (used by elinks --dump), detect the
second cell of double-cell (aka fullwidth) characters by comparing to
UCS_NO_CHAR, like add_document_to_string does. Don't use
unicode_to_cell for this any more.
Also, ignore the colors and attributes of the second cell; don't
output any escape sequences for them.
In start_document_refresh, use register_bottom_half instead of
install_timer if the timeout is 0 because install_timer asserts that it is
given a delay greater than 0.
Add a test case, test/refresh-0timeout.html. Note that
document.browse.minimum_refresh_time must be set to 0 to reproduce the
assertion failure.
Add a 'Search contents' button to the cache manager that searches
through the cache items' data rather than their metadata.
Add match_cache_entry_contents.
Add push_cache_hierbox_search_button and
push_cache_hierbox_search_contents_button, which call
push_hierbox_search_button after setting box->ops to either
cache_entry_listbox_ops or cache_entry_listbox_ops_match_contents,
respectively, which define the appropriate match callback for the
hierbox search code.
Add strlcasestr, used in the new match_cache_entry_contents routine.
Add an "Invalidate" button to the cache manager, which is useful when
a cache entry is locked and cannot be deleted but one does not want
ELinks to use that cache entry.
This is especially useful for showing that neither dump_truecolor_utf8
nor dump_truecolor_unibyte modifies its static color[] variable and it
therefore does not matter whether those functions use the same array
or not.
With all the comments and macros needed for this, the source files
don't become much shorter, but anyway I hope they'll be easier to
maintain this way.
This code was included in four variants of dump_to_file().
Move it to a new function dump_references() and make dump_to_file()
then call that. This makes the code size a little smaller.
The time cost will be negligible.
Instead of having four separate function definitions, have just one
sprinkled with #ifdefs, and #include that four times. The purpose
being to make it clearer which parts of these functions are identical
and which ones differ.
As a side effect, this change makes ELinks ignore --dump-color-mode
when dumping in UTF-8. Colourful UTF-8 dumping has not been
implemented and the fallback is now different from before.
Avoid compilation error with GNUTLS 1.2.9:
/home/Kalle/src/elinks-0.12/src/network/ssl/ssl.c:258: error: implicit declaration of function ‘gnutls_priority_set_direct’
If the function is not available, use gnutls_set_default_priority instead.
Perhaps it'll work with bugzilla.novell.com, perhaps not.
- gnutls_handshake_set_private_extensions: Do not enable private cipher
suites that might not be supported by anything other than GNUTLS.
The GNUTLS 2.8.0 documentation notes that enabling these extensions
can cause interoperability problems.
- gnutls_set_default_priority: Explicitly disable OpenPGP certificates.
- gnutls_certificate_type_set_priority: Do not enable OpenPGP certificates.
The GNUTLS 2.8.0 documentation notes that OpenPGP certificate support
requires libgnutls-extra. Because libgnutls-extra 2.2.0 and later are
under GPLv3-or-later and thus not GPLv2 compatible, ELinks doesn't use
libgnutls-extra, so OpenPGP certificates didn't work anyway.
- gnutls_server_name_set: Do not tell the server the hostname from the URL.
This was supposed to let the server choose the appropriate certificate
for each name-based virtual host, but ELinks actually always sent just
"localhost", so it didn't work anyway. This will have to be revisited
when ELinks is changed to actually verify the subject name from the
server's certificate (ELinks bug 1024).
These changes should help ELinks negotiate SSL with bugzilla.novell.com.
[NEWS and commit message by me. --KON]
Yet another valiant wack at the beast. This one violates abstractions
a little less deeply, so maybe it will work better.
The last attempt caused a crash when a tab was cloned after the tab's
loading had been aborted.
(cherry picked from commit 76377d9714)
Kalle reported that after commit 5c96d430c9,
ELinks would crash if the document in the old tab was still loading when a
new tab was opened. The problem was that the new session's download.data
pointer was not updated to point to the session as doc_loading_callback
expects.
Instead of just calling render_document_frames, set up the download and
call load_uri.
(cherry picked from commit d6116ca83a)
In setup_session, use copy_location, add_to_history, and
render_document_frames instead of goto_uri and copy_vs to copy the base
tab's view state. By avoiding goto_uri, setup_session now bypasses MIME
checks, form post confirmations, malicious URL checks, and so on when
copying the base tab's current location and view state to the new tab,
so the new tab should get exactly what was loaded in the base tab.
This fixes bug 765: Opening a new tab can ask about the document of the
previous tab.
(cherry picked from commit 5c96d430c9)
Conflicts:
src/session/session.c:
Both elinks-0.12 and master had the ses->doc_view->vs
= vs assignment, but only elinks-0.12 had vs->doc_view
= ses->doc_view as well. Also, struct connection_state
had been added after the original patch.
I am not hooking these to "make test", for two reasons:
1. utf8_step_forward is inside #ifdef CONFIG_UTF8 and I don't see
how to make tests conditional on such options.
2. test/libtest.sh was copied from Git, which is under GPLv2-only.
Adding more dependencies on it could make ELinks more difficult
to relicense under GPLv2-or-later.
In the task.c line 517 there is:
if (is_in_progress_state((*download_p)->state)) {
if (have_location(ses))
*download_p = &cur_loc(ses)->download;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Here the download was changed. download->data and download->callback
were NULL after the assignment, but later in loading_callback
only download->callback had new value. download->data was still NULL.
Yet another valiant wack at the beast. This one violates abstractions
a little less deeply, so maybe it will work better.
The last attempt caused a crash when a tab was cloned after the tab's
loading had been aborted.
In load_uri, if there is no valid cache entry and no existing connection
for the requested URI, make one last check in case there is an
incomplete cache entry.
Simplify the end-of-line check in get_search_region_from_search_nodes by
relying on the fact that the n member of an instance of struct search
that marks the end of a line will be 0.
Allow searching on the last character of the document. Plain-text searches
already match on the last character as long as it isn't the first character
of a match, and regular-expression searches match on the last character if
the search pattern is longer than 1 character, so the problem addressed by
this commit is very much a corner case.
This commit reverts a portion of commit
fd15049622594d151104d43917984c7ce10993e6 (CVS revision 1.17).
text_typeahead_handler: Document that passing -2 for action_id will cause
a search without error reporting. This behaviour is unintentionally the
current behaviour of text_typeahead_handler, but now it is documented so
that it can be used.
input_line_event_handler: When rewinding, pass -2 for the action_id
parameter to the handler instead of passing again whatever action led to
the rewinding.
The old behavior of input_line_event_handler was particularly problematic
with the search-toggle-regex action and the text_typeahead_handler handler:
input_line_event_handler would call the handler with
ACT_EDIT_SEARCH_TOGGLE_REGEX, and the handler would toggle the setting and
perform the search again; then if the search string no longer matched
anything, the handler would return INPUT_LINE_REWIND to
input_line_event_handler, which would rewind and call the handler with
ACT_EDIT_SEARCH_TOGGLE_REGEX again, thus toggling the option back to the
original setting.
With the new behaviour, input_line_event_handler will not repeat the same
action when re-invoking the handler; in the above example with
search-toggle-regex, the search string will simply be rewound until it
matches with the new setting.
When a link had an onClick event handler that changed the current
document and that link was clicked, ELinks would follow the current link
of the document displayed after executing the handler instead of the
link that was clicked.
Factor goto_link out of goto_current_link.
Use goto_link instead of goto_current_link in activate_link to ensure that
the link that is passed in by enter() is followed.
Hierarchical listboxes draw items with upper-left corner, lower-left
corner, or horizontal border characters to indicate whether a given item is
the first item in a listbox, the last, or any other, respectively.
However, the wrong character can be drawn if there are invisible items: if
an item is the first (or last) visible item but there is an invisible item
before (or after) it, it will be drawn with a horizontal border character,
not a corner.
This patch fixes that problem using traverse_listbox_items_list in
display_listbox_item to ignore invisible items when determining whether
an item is either the first or the last among its siblings.
Drop special handling of ctrl-l in handle_interlink_event.
To make sure that the 'redraw' action works everywhere, first modify
menu_kbd_handler and mainmenu_kbd_handler to handle ACT_MENU_REDRAW; and
second, drop the ACTION_REQUIRE_VIEW_STATE flag from the 'redraw' action in
the 'main' keymap so that it works even if there is no document loaded.
Ctrl-l is already bound to 'redraw' in all keymaps by default, so the
current default behaviour is preserved.
Add 'Italic' checkbox to Terminal options dialog box.
Enable italic text by default for rxvt-unicode (and also enable
frames, 88-colour mode, and underline).
Kalle reported that after commit 5c96d430c9,
ELinks would crash if the document in the old tab was still loading when a
new tab was opened. The problem was that the new session's download.data
pointer was not updated to point to the session as doc_loading_callback
expects.
Instead of just calling render_document_frames, set up the download and
call load_uri.
This check used to be in src/elinks.h. Move it to configure.in so
that (1) the result can be logged and (2) ELinks won't even link with
TRE if wchar_t prevents its use.
Also, rename HAVE_TRE_REGEX_H to CONFIG_TRE, to reflect that it is not
always defined if the header exists.
At the end of the destroy_vs there two assignments vs->doc_view->vs = NULL and
vs->doc_view = NULL. In the setup_session the copy_vs left the vs "unbound"
with any variable. At least one of these two is wrong.
Fix this error when configured with --enable-debug --disable-utf-8:
[CC] src/bfu/text.o
cc1: warnings being treated as errors
/home/Kalle/src/elinks-0.13/src/bfu/text.c: In function ‘dlg_format_text_do’:
/home/Kalle/src/elinks-0.13/src/bfu/text.c:220: error: unused variable ‘term’
Fix this error when configured with --enable-debug --disable-utf-8:
[CC] src/bfu/button.o
cc1: warnings being treated as errors
/home/Kalle/src/elinks-0.13/src/bfu/button.c: In function ‘dlg_format_buttons’:
/home/Kalle/src/elinks-0.13/src/bfu/button.c:122: error: unused variable ‘term’
When setting up default values for terminal options, use named
constants like TERM_VT100 or COLOR_MODE_16, rather than plain integers
like 1. This is just to make the source code easier to read and
perhaps more resistant to future bugs. The binary should not change.
If globhist_simple_search ran out of memory in stracpy(search_url), it
could leave gh_last_searched_title pointing to freed memory and cause
a crash in the next call. Fix by not freeing gh_last_searched_title.
It is then possible to have gh_last_searched_title and
gh_last_searched_url pointing to strings from different searches;
but that was already possible if stracpy(search_title) failed.
Because this bug occurs only in out-of-memory situations and I don't
think ELinks in general has been properly tested in those, the fix is
perhaps not worth mentioning in NEWS and backporting to elinks-0.11.
If the parent parameter of get_dom_node_list_index referred to a node
that did not have children, then get_dom_node_list called by it could
return the address of a null pointer, and get_dom_node_list_index would
then pass that null pointer to get_dom_node_list_pos, which would crash.
That would be the same kind of crash as the one in get_dom_node_child.
It never happened in practice though: because all calls are in the form
get_dom_node_list_index(node->parent, node), the list must contain at
least the given node, and the pointer cannot be null. The documentation
of get_dom_node_list_index allows arbitrary nodes as arguments however,
so it's best to add a check.
struct dom_node contains a union that contains various structs that
have members of type struct dom_node * in them.
get_dom_node_list_by_type returns the address (struct dom_node **) of
one of those members, or NULL. However the member itself can also be
NULL if no nodes have been added to the list and the list has thus not
yet been allocated. (add_to_dom_node_list lazily allocates the lists.)
get_dom_node_child did not expect a null pointer there and crashed, as
shown in bug 1071. Fix by adding a check so that it treats a NULL list
as an empty list.
C99 6.7.4p3 and 6.7.4p6 set some constraints on what can be done in
inline functions and how they can be declared. In particular, any
function declared inline must also be defined in the same translation
unit. To comply with that, remove inline specifiers from function
declarations in header files when the functions are not also defined
in those header files.
Sun Studio 11 on Solaris 9 is stricter than C99 and does not allow
references to static identifiers in extern inline functions. Make the
configure script detect this and define NONSTATIC_INLINE accordingly
in config.h. Then use that in the definitions of all non-static
inline functions.
Document the restrictions and this scheme in doc/hacking.txt.
Documentation strings of most options used to contain a "\n" at the
end of each source line. When the option manager displayed these
strings, it treated each "\n" as a hard newline. On 80x24 terminals
however, the option description window has only 60 columes available
for the text (with the default setup.h), and the hard newlines were
further apart, so the option manager wrapped the text a second time,
resulting in rather ugly output where long lones are interleaved with
short ones. This could also cause the text to take up too much
vertical space and not fit in the window.
Replace most of those hard newlines with spaces so that the option
manager (or perhaps BFU) will take care of the wrapping. At the same
time, rewrap the strings in source code so that the source lines are
at most 79 columns wide.
In some options though, there is a list of possible values and their
meanings. In those lists, if the description of one value does not
fit in one line, then continuation lines should be indented. The
option manager and BFU are not currently able to do that. So, keep
the hard newlines in those lists, but rewrap them to 60 columns so
that they are less likely to require further wrapping at runtime.
test_search() was supposed to compare bookmark titles with
strcasestr(), but in commit 311d95358d
"bug 153, 1066: Convert bookmarks to/from UTF-8 when searching."
on 2009-02-08, I inadvertently changed that to strcasecmp(), even
while adding a comment about why strcasestr() is needed. strcasestr()
returns non-NULL if the strings match, and strcasecmp() returns
nonzero if they differ, so the search didn't work at all.
This reverts commit b94657869b.
I don't know where I got the idea that JS_SetErrorReporter affects the
entire JSRuntime, rather than only the provided JSContext. The people
on #jsapi say it has never worked that way.
In utf8_to_jsstring, do not free the string that is passed to
JS_NewUCString if the latter is successful; if it is, SpiderMonkey
handles the memory from then on.
Use libc routines instead of ELinks's routines to allocate and free the
string so that ELinks's memory debugging code does not try to keep track
of it after it has been handed to SpiderMonkey.
This commit fixes a bug introdued in
97d72d15a0.
In utf8_to_jsstring, do not free the string that is passed to
JS_NewUCString if the latter is successful; if it is, SpiderMonkey
handles the memory from then on.
Use libc routines instead of ELinks's routines to allocate and free the
string so that ELinks's memory debugging code does not try to keep track
of it after it has been handed to SpiderMonkey.
This commit fixes a bug introdued in
97d72d15a0.
This makes the option manager display it much better on an 80x24
terminal.
Alternatively, the "\n" newline characters within paragraphs could
have been removed entirely. ELinks would then have line-wrapped the
text to the appropriate width in the info window of the option
manager, but unfortunately not in --config-help.
AFAIK, all bugs in it have been fixed. Some bugs may still be lurking
but they are more likely to get caught if compression is enabled.
I also replaced COMP_NOTE with static text because xgettext does not
support macros in the argument of N_.
(cherry picked from commit 3a9b5d091d)
Check the return value of get_opt_rec on "document.browse.search.regex"
before dereferencing it. The option is not there if regular expression
support is disabled at build time.
This commit fixes a bug introduced in commit
b2d51c75ff0d6c52a4f6a2761801beb641cba3a2.
Check the return value of get_opt_rec on "document.browse.search.regex"
before dereferencing it. The option is not there if regular expression
support is disabled at build time.
This commit fixes a bug introduced in commit
b2d51c75ff0d6c52a4f6a2761801beb641cba3a2.
In src/session/task.c, if ses_goto() was going to ask the user to
confirm, it did:
task->session_task.target.frame = null_or_stracpy(target_frame);
It added the struct task to a memory_list, so the structure was freed
when the message box was closed. The target frame string was however
never freed. To fix this leak, add the target frame string to the
memory_list too.
Alternatively, this could have been fixed by making post_yes() and
post_no() free the string. It is however a bit better to use the
memory_list because msg_box() frees that even if it is unable to
display the message box.
In bug 1067, dom_rss_pop_document() freed a node with done_dom_node()
even though call_dom_node_callbacks() was still using that node. This
made call_dom_node_callbacks() read a function pointer from beyond the
end of an array and call that. Add assertions to detect out-of-range
node types, and comments to warn about the bug.
Debian libmozjs-dev 1.9.0.4-2 has JS_ReportAllocationOverflow but
js-1.7.0 reportedly hasn't. Check at configure time whether that
function is available. If not, use JS_ReportOutOfMemory instead.
Reported by Witold Filipczyk.
I didn't read the code of the tre library, but I suppose that when sizes of
wchar_t and unicode_val_T are equal it will work fine.
[ From bug 1060 attachment 508. --KON ]
When the user tells ELinks to search for a regexp, ELinks 0.11.0
passes the regexp to regcomp() and the formatted document to
regexec(), both in the terminal charset. This works OK for unibyte
ASCII-compatible charsets because the regexp metacharacters are all in
the ASCII range. And ELinks 0.11.0 doesn't support multibyte or
ASCII-incompatible (e.g. EBCDIC) charsets in terminals, so it is no
big deal if regexp searches fail in such locales.
ELinks 0.12pre1 attempts to support UTF-8 as the terminal charset if
CONFIG_UTF8 is defined. Then, struct search contains unicode_val_T c
rather than unsigned char c, and get_srch() and add_srch_chr()
together save UTF-32 values there if the terminal charset is UTF-8.
In plain-text searches, is_in_range_plain() compares those values
directly if the search is case sensitive, or folds them to lower case
if the search is case insensitive: with towlower() if the terminal
charset is UTF-8, or with tolower() otherwise. In regexp searches
however, get_search_region_from_search_nodes() still truncates all
values to 8 bits in order to generate the string that
search_for_pattern() then passes to regexec(). In UTF-8 locales,
regexec() expects this string to be in UTF-8 and can't make sense of
the truncated characters. There is also a possible conflict in
regcomp() if the locale is UTF-8 but the terminal charset is not, or
vice versa.
Rejected ways of fixing the charset mismatches:
* When the terminal charset is UTF-8, recode the formatted document
from UTF-32 to UTF-8 for regexp searching. This would work if the
terminal and the locale both use UTF-8, or if both use unibyte
ASCII-compatible charsets, but not if only one of them uses UTF-8.
* Convert both the regexp and the formatted document to the charset of
the locale, as that is what regcomp() and regexec() expect. ELinks
would have to somehow keep track of which bytes in the converted
string correspond to which characters in the document; not entirely
trivial because convert_string() can replace a single unconvertible
character with a string of ASCII characters. If ELinks were
eventually changed to use iconv() for unrecognized charsets, such
tracking would become even harder.
* Temporarily switch to a locale that uses the charset of the
terminal. Unfortunately, it seems there is no portable way to
construct a name for such a locale. It is also possible that no
suitable locale is available; especially on Windows, whose C library
defines MB_LEN_MAX as 2 and thus cannot support UTF-8 locales.
Instead, this commit makes ELinks do the regexp matching with regwcomp
and regwexec from the TRE library. This way, ELinks can losslessly
recode both the pattern and the document to Unicode and rely on the
regexp code in TRE decoding them properly, regardless of locale.
There are some possible problems though:
1. ELinks stores strings as UTF-32 in arrays of unicode_val_T, but TRE
uses wchar_t instead. If wchar_t is UTF-16, as it is on Microsoft
Windows, then TRE will misdecode the strings. It wouldn't be too
hard to make ELinks convert to UTF-16 in this case, but (a) TRE
doesn't currently support UTF-16 either, and it seems possible that
wchar_t-independent UTF-32 interfaces will be added to TRE; and (b)
there seems to be little interest on using ELinks on Windows anyway.
2. The Citrus Project apparently wanted BSD to use a locale-dependent
wchar_t: e.g. UTF-32 in some locales and an ISO 2022 derivative in
others. Regexp searches in ELinks now do not support the latter.
[ Adapted to elinks-0.12 from bug 1060 attachment 506.
Commit message by me. --KON ]
In src/bookmarks/dialogs.c, do_add_bookmark() gets the title and URL
in the terminal charset and needs to know which one that is. When a
bookmark is being added, save the struct terminal * to dialog.udata2
and read the charset from there. When a bookmark is being edited,
dialog.udata2 is needed for the struct bookmark *, but there we always
have the parent struct dialog_data * in dialog.udata and can get the
terminal from that.
These functions now expect or return strings in UTF-8:
delete_folder_by_name (sneak in a const, too), bookmark_terminal_tabs,
open_bookmark_folder, and get_auto_save_bookmark_foldername_utf8 (new
function).
When setting the title or URL of a bookmark from SMJS user scripting,
use update_bookmark() instead of writing directly to struct bookmark.
It triggers the bookmark-update event and sets the bookmarks_dirty
flag.
SpiderMonkey uses UTF-16 and the strings in struct bookmark are in
UTF-8. Previously, the conversions behaved as if the strings had been
in ISO-8859-1.
SpiderMonkey also supports JS_SetCStringsAreUTF8(), which would make
the existing functions convert between UTF-16 and UTF-8, but that
effect is global so I dare not enable it yet. Besides, I don't know
if that function works in all the SpiderMonkey versions that ELinks
claims to work with.
This also makes the bookmark-update event carry strings in UTF-8.
The only current consumer of that event is bookmark_change_hook(),
which ignores the strings, so no changes are needed there.
When the file is being read, Expat provides the strings to ELinks in
UTF-8, so ELinks can put them in struct bookmark without conversions.
Make sure gettext returns any placeholder strings in UTF-8, too.
Replace '\r' with ' ' in bookmark titles and URLs.
When the file is being written, put encoding="UTF-8" in the XML
declaration, and then write out the strings from struct bookmark
without character set conversions. Do replace some characters
with entity references though, by calling add_html_to_string().
When ELinks is parsing an XML element in from an XBEL bookmark file,
it collects the attributes of the element to the current_node->attrs
list. Previously, struct attributes had room for one string only:
the last element of current_node->attrs was the name of the first
attribute, and it was preceded by the value of the first attribute,
the name of the second attribute, the value of the second attribute,
and so on. However, when get_attribute_value() was looking for a
given name, it compared the values as well. So, if you had for
example <bookmark id="href" href="http://elinks.cz/">, then
get_attribute_value("href") would incorrectly return "href".
To fix this confusion, store values in the new member
attributes.value, rather than in attributes.name.
Change test/imgmap2.html so it can be used for testing this too.
Debian Iceweasel 3.0.4 does not appear to support such external
client-side image maps. Well, that's one place where ELinks is
superior, I guess. There might be a security problem though if ELinks
were to let scripts of the referring page examine the links in the
image map.
When ELinks runs in an X11 terminal emulator (e.g. xterm), or in GNU
Screen, it tries to update the title of the window to match the title
of the current document. To do this, ELinks sends an "OSC 1 ; Pt BEL"
sequence to the terminal. Unfortunately, xterm expects the Pt string
to be in the ISO-8859-1 charset, making it impossible to display e.g.
Cyrillic characters. In xterm patch #210 (2006-03-12) however, there
is a menu item and a resource that can make xterm take the Pt string
in UTF-8 instead, allowing characters from all around the world.
The downside is that ELinks apparently cannot ask xterm whether the
setting is on or off; so add a terminal._template_.latin1_title option
to ELinks and let the user edit that instead.
Complete list of changes:
- Add the terminal._template_.latin1_title option. But do not add
that to the terminal options window because it's already rather
crowded there.
- In set_window_title(), take a new codepage argument. Use it to
decode the title into Unicode characters, and remove only actual
control characters. For example, CP437 has graphical characters in
the 0x80...0x9F range, so don't remove those, even though ISO-8859-1
has control characters in the same range. Likewise, don't
misinterpret single bytes of UTF-8 characters as control characters.
- In set_window_title(), do not truncate the title to the width of the
window. The font is likely to be different and proportional anyway.
But do truncate before 1024 bytes, an xterm limit.
- In struct itrm, add a title_codepage member to remember which
charset the master said it was going to use in the terminal window
title. Initialize title_codepage in handle_trm(), update it in
dispatch_special() if the master sends the new request
TERM_FN_TITLE_CODEPAGE, and use it in most set_window_title() calls;
but not in the one that sets $TERM as the title, because that string
was not received from the master and should consist of ASCII
characters only.
- In set_terminal_title(), convert the caller-provided title to
ISO-8859-1 or UTF-8 if appropriate, and report the codepage to the
slave with the new TERM_FN_TITLE_CODEPAGE request. The conversion
can run out of memory, so return a success/error flag, rather than
void. In display_window_title(), check this result and don't update
caches on error.
- Add a NEWS entry for all of this.
This simplifies the callers a little and may help implement
simultaneous support for different charsets on different terminals
of the same type (bug 1064).
look_for_link() used to return 0 both when it found the closing </MAP>
tag, and when it hit the end of the file. In the first case, it also
added *menu to the memory_list; in the second case, it did not. The
caller get_image_map() supposedly distinguished between these cases by
checking whether pos >= eof, and freed *menu separately if so.
However, if the </MAP> was at the very end of the HTML file, so that
not even a newline followed it, then look_for_link() left pos == eof
even though it had found the </MAP> and added *menu to the memory_list.
This made get_image_map() misinterpret the result and mem_free(*menu)
even though *menu had already been freed as part of the memory_list;
thus the crash.
To fix this, make look_for_link() return -1 instead of 0 if it hits
EOF without finding the </MAP>. Then make get_image_map() check the
return value instead of comparing pos to eof. And add a test case,
although not an automated one.
Alternatively, look_for_link() could have been changed to decrement
pos between finding the </MAP> and returning 0. Then, the pos >= eof
comparison in get_image_map() would have been false. That scheme
would however have been a bit more difficult to understand and
maintain, I think.
Reported by Paul B. Mahol.
(cherry picked from commit a2404407ce)
Before this patch, if you first moved the cursor to link X with
move-cursor-up and similar actions, and then clicked link Y with the
mouse, ELinks would activate link X, i.e. not the one you clicked.
This happened because the NAVIGATE_CURSOR_ROUTING mode was left
enabled and made ELinks ignore the doc_view->vs->current_link
member that ELinks had updated according to the click.
Make ELinks return the session to NAVIGATE_LINKWISE mode, so that
the update takes effect.
Reported by Paul B. Mahol.
(cherry picked from commit 4086418069)
In get_entity_string and point_intersect, do not initialise arrays with
static storage duration to zero; the C standard states that such objects
are automatically initialised to zero.
A style-sheet containing the string "url( )" with 1 or more characters of
whitespace in between the parentheses triggered an assertion failure in
scan_css_token.
scan_css_token would find the left parenthesis, find the right parenthesis,
and then scan forwards from the left parenthesis for a non-whitespace
character to find the start of the URL and backwards from the right
parenthesis to find the end of the URL. If there were whitespace and
nothing else, the start would be past the end, the routine would compute a
negative length for the URL, and then the routine would trigger an
assertion failure. Now the routine simply enforces a lower bound of length
0 for the URL.
This patch fixes an issue whereby a newline character appearing within
a hidden input field is incorrectly reinterpreted as a space character.
The patch handles almost all cases, and includes a test case.
15/18 tests pass, but the remainder currently fail due to the fact
that ELinks does not currently support textarea scripting.
c_strcasecmp and c_strncasecmp were taken from GNU coreutils 6.9,
which is copyrighted by the Free Software Foundation and licensed
under GNU GPL version 2 or later. It seems the programs in coreutils
do not normally read commands interactively. So, including coreutils
code in an interactive program such as ELinks could trigger GPLv2
section 2. c), which would require ELinks to display a copyright
notice and a warranty disclaimer each time it is started. Rewrite
those functions to remove the FSF-copyrighted code and make ELinks
not a work based on GNU coreutils.
Avoiding FSF code has the additional benefit that we won't have to ask
FSF for permission if we want to add a licence exception that allows
linking ELinks with OpenSSL. So it seems a good idea even if my
interpretation of GPLv2 2. c) is overly strict. I haven't checked
though whether there are other FSF-copyrighted portions in ELinks.
src/config/kbdbind.c (parse_keystroke): If the user types "Ctrl-i",
it should mean "Ctrl-I" rather than "Ctrl-İ", because the Ctrl-
combinations are only well known for ASCII characters. This does not
matter in practice though, because src/terminal/kbd.c converts 0x09
to (KBD_MOD_NONE, KBD_TAB) and not to (KBD_MOD_CTRL, 'I').
src/osdep/beos/beos.c (get_system_env): Changing the locale does not
affect the TERM environment variable, I think, so it should not affect
the interpretation either.
Add #include directives to fix these errors:
[CC] src/intl/gettext/l10nflist.o
cc1: warnings being treated as errors
.../src/intl/gettext/l10nflist.c: In function ‘_nl_normalize_codeset’:
.../src/intl/gettext/l10nflist.c:352: error: implicit declaration of function ‘c_tolower’
[CC] src/dom/css/scanner.o
cc1: warnings being treated as errors
In file included from .../src/dom/scanner.h:4,
from .../src/dom/css/scanner.h:4,
from .../src/dom/css/scanner.c:12:
.../src/dom/string.h: In function ‘dom_string_casecmp’:
.../src/dom/string.h:34: error: implicit declaration of function ‘c_strncasecmp’
