editor/parser refactoring

jgangemi wrote on Sun Aug 13 04:35:30 MEST 2006:

i was curious what the state of this was - i've fixed the folding issue
that would cause everything to be unfolded when the reconcilliation thread
ran, however i've uncoverred another bug and/or limitation of the parser.

for instance, if i add new subroutines to a module one after another, everything
works correctly - however, if i add a new method in between two folded methods,
the last folded method becomes unfolded as soon as the word "sub" is detected
in the document.

it seems that the document offset is calculated as the start of the new
subroutine to the next complete subroutine or the end of the file is reached.

the same applies when writing perldoc tags (they cause any subroutines below
them be expanded as well), etc.

you said previously that most of the syntax highlighting issues were fixed,
so perhaps my issues are addressed as well.

jgangemi wrote on Sun Aug 13 22:30:12 MEST 2006:

apparently my issue caused by the parser :)

still interested in the status of this though...

jploski wrote on Sun Aug 20 20:15:01 MEST 2006:

I just tried the following using 'stable': enter two subroutines, fold them,
then insert a third one between them, save. The folds of the two subs stay
unchanged.

You wrote:

> it seems that the document offset is calculated
> as the start of the new subroutine to the next
> complete subroutine or the end of the file is
> reached. 

I lack the context to comment on it (which document offset, calculated where/when?)

As far as the state of parser refactorings is concerned: I assume you are
referring to the transition from 0.3.12 to 0.4. This has fixed many of the
reported problems (hence closed bugs). I feel the parser is going to be
a never-ending story. The new parser, though faster and more accurate than
the original JEdit-based one, uses rather simple heuristics rather than
a language specification (which does not even exist for Perl - except for
its own source code). Also, the number of automated regression tests is
still humble (see syntax.pl in org.epic.perleditor-test and TestPerlMultiLexer
there. I also run the parser against *.pm on my machine as a sort of smoke
test). The idea is that we keep watching the common usage and add new heuristics
as problems are reported.

jgangemi wrote on Sun Aug 20 20:27:55 MEST 2006:

i have since figured out that it is the reconcilliation thread that causes
the unfolding behavior. as long as the subroutine is completed before it
runs, the other subroutine won't unfold. 

here is an example of the what i mean for the offset calculation:

if foo was folded and i started entering bar, and paused before the closing
} was added, foo would expand.

before:

+foo

after:

sub bar
{

sub foo
{
}

the jdt folding seems to work in the same way, although it has it's own
quirks.

i'm still confused on the parser changed. i thought you originally said
you had a new parser implementation that was more tightly coupled to antlr
sitting locally on your box. if that is still correct, i was wondering when
those changes would be checked in to try out (even if on a private branch).

jploski wrote on Sun Aug 20 21:07:32 MEST 2006:

I have no new parser implementation waiting. I can't remember which message
you might be referring to, sorry.

jgangemi wrote on Mon Aug 21 16:10:24 MEST 2006:

i was referring to this thread: https://sourceforge.net/forum/forum.php?thread_id=1391662&forum_id=258689