Announcement

Collapse
No announcement yet.

structured return values.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    structured return values.



    had a thought earlier today related to structures as return values...

    When I run into situations where I want a function to modify multiple
    values, I tend to add those values as parameters, and pass them by VAR
    then let the function modify them.

    This has a few side effects (some good, some bad)...

    not intuitive, you can't tell that the assign is happening simply by
    looking at the function call.

    if the function doesn't return a value, you find yourself using
    MvEVAL's, which just don't seem right.

    the function can still return a value, which I tend to use as the
    'l.ok' success/fail value.

    Now I've been using these approaches for some time. Part of the
    reason for that, was a speed issue. It seemed that when you had very
    large structures, the code ran faster (uncompiled) if you passed them
    by var instead of passing them back as a return value. I'm not sure
    whether this is still an issue or not, and should probably run some
    tests to find out.

    Assuming speed is no longer an issue (compiled at least), I'm thinking
    of changing it to structured return values. Dunno why it never
    crossed my mind, and not sure how well it would work, but I'm thinking
    if I use some standard naming convention, it could work.

    Can still return an 'ok' value, just as a branch:
    l.ret:ok

    If it's ok, stuff any data you have on a data branch:
    l.ret:data

    If you have errors, you can stick them on independent branches --
    which allows you to track multiple errors, and/or respond to different
    errors different ways.
    l.ret:error:MvOpen
    l.ret:error:EOF
    l.ret:error:Fexists
    l.ret:error:DataEntry

    It may also pay to make it a structured array, due to miva's lack of
    'reflection' (ie: not easy to ask a structure "what branches do you
    have?" - at least not in 4.x and under).

    The error branch is the beauty of it, as it's let's you seperate error
    handling from the core functions. All they do is report the error,
    it's up to the calling function to decide what to do - which could
    include ignoring them. Of course, that does mean the burden shifts,
    but that can be handled with some generic error handling (which can
    vary by app, or by function call).

    In Merchant's db.mv code, there are a lot of functions that can't be
    used 'flexibly' because they cause the script to error out.=20
    Sometimes... "Category_Find_Code" is just a 'check', but if I need to
    check to see if a category code exists, I can't use that function due
    to it's hard coded error check.

    anyway, just thinking out loud, thought I'd share it.

    --=20
    Bill Guindon (aka aGorilla)



    #2
    structured return values.



    Hi Bill,

    Thanks for sharing your ideas and your research on the list. I hope other
    ppl on that list ("community") will follow you example. There are just a few
    (but very valuable) who do that.. but in my opinion that's not enough..

    Thanks again
    Claudiu Bischoff

    -----Original Message-----
    From: [email protected]
    [mailto:[email protected]]On Behalf Of Bill Guindon
    Sent: lundi 16 mai 2005 03:17
    To: Miva Users
    Subject: [meu] structured return values.


    had a thought earlier today related to structures as return values...

    When I run into situations where I want a function to modify multiple
    values, I tend to add those values as parameters, and pass them by VAR
    then let the function modify them.

    This has a few side effects (some good, some bad)...

    not intuitive, you can't tell that the assign is happening simply by
    looking at the function call.

    if the function doesn't return a value, you find yourself using
    MvEVAL's, which just don't seem right.

    the function can still return a value, which I tend to use as the
    'l.ok' success/fail value.

    Now I've been using these approaches for some time. Part of the
    reason for that, was a speed issue. It seemed that when you had very
    large structures, the code ran faster (uncompiled) if you passed them
    by var instead of passing them back as a return value. I'm not sure
    whether this is still an issue or not, and should probably run some
    tests to find out.

    Assuming speed is no longer an issue (compiled at least), I'm thinking
    of changing it to structured return values. Dunno why it never
    crossed my mind, and not sure how well it would work, but I'm thinking
    if I use some standard naming convention, it could work.

    Can still return an 'ok' value, just as a branch:
    l.ret:ok

    If it's ok, stuff any data you have on a data branch:
    l.ret:data

    If you have errors, you can stick them on independent branches --
    which allows you to track multiple errors, and/or respond to different
    errors different ways.
    l.ret:error:MvOpen
    l.ret:error:EOF
    l.ret:error:Fexists
    l.ret:error:DataEntry

    It may also pay to make it a structured array, due to miva's lack of
    'reflection' (ie: not easy to ask a structure "what branches do you
    have?" - at least not in 4.x and under).

    The error branch is the beauty of it, as it's let's you seperate error
    handling from the core functions. All they do is report the error,
    it's up to the calling function to decide what to do - which could
    include ignoring them. Of course, that does mean the burden shifts,
    but that can be handled with some generic error handling (which can
    vary by app, or by function call).

    In Merchant's db.mv code, there are a lot of functions that can't be
    used 'flexibly' because they cause the script to error out.
    Sometimes... "Category_Find_Code" is just a 'check', but if I need to
    check to see if a category code exists, I can't use that function due
    to it's hard coded error check.

    anyway, just thinking out loud, thought I'd share it.

    --
    Bill Guindon (aka aGorilla)


    Comment


      #3
      structured return values.



      Passing references or pointers in function arguments is a common technique
      in many languages. It is of course much more efficient, especially at more
      complex arguments like structures or arrays, because it avoids creating the
      temporary local variables and copying the original content to the stack (or
      standard operating memory in some cases) and then back again. References not
      only save all those extra CPU instructions needed for the creation of the
      temporary variables, but it also avoids blocking additional memory.

      The disadvantage of breaking the intuitivity and readability is real, but
      can be partially reduced by consistently following "Hungarian" naming
      conventions of the functions: using a specific prefixes at those functions,
      so that the difference between a function with plain arguments and a
      function using (and possibly modifying) references is apparent on the first
      look.

      It certainly does not mean that references should be used everywhere instead
      of plain arguments. Understanding what happens on lower levels (OS / BIOS /
      Assembler / CPU ...) certainly helps very much when programming in high
      level languages like Miva Script too, but generally you can consider that
      passing arrays and structures as references is typically much more
      efficient.

      Ivo
      http://mivo.truxoft.com




      -----Original Message-----
      From: Claudiu

      Hi Bill,

      Thanks for sharing your ideas and your research on the list. I hope other
      ppl on that list ("community") will follow you example. There are just a few
      (but very valuable) who do that.. but in my opinion that's not enough..

      Thanks again
      Claudiu Bischoff

      -----Original Message-----
      From: [email protected]
      [mailto:[email protected]]On Behalf Of Bill Guindon
      Sent: lundi 16 mai 2005 03:17
      To: Miva Users
      Subject: [meu] structured return values.


      had a thought earlier today related to structures as return values...

      When I run into situations where I want a function to modify multiple
      values, I tend to add those values as parameters, and pass them by VAR
      then let the function modify them.

      This has a few side effects (some good, some bad)...

      not intuitive, you can't tell that the assign is happening simply by
      looking at the function call.

      if the function doesn't return a value, you find yourself using
      MvEVAL's, which just don't seem right.

      the function can still return a value, which I tend to use as the
      'l.ok' success/fail value.

      Now I've been using these approaches for some time. Part of the
      reason for that, was a speed issue. It seemed that when you had very
      large structures, the code ran faster (uncompiled) if you passed them
      by var instead of passing them back as a return value. I'm not sure
      whether this is still an issue or not, and should probably run some
      tests to find out.

      Assuming speed is no longer an issue (compiled at least), I'm thinking
      of changing it to structured return values. Dunno why it never
      crossed my mind, and not sure how well it would work, but I'm thinking
      if I use some standard naming convention, it could work.

      Can still return an 'ok' value, just as a branch:
      l.ret:ok

      If it's ok, stuff any data you have on a data branch:
      l.ret:data

      If you have errors, you can stick them on independent branches --
      which allows you to track multiple errors, and/or respond to different
      errors different ways.
      l.ret:error:MvOpen
      l.ret:error:EOF
      l.ret:error:Fexists
      l.ret:error:DataEntry

      It may also pay to make it a structured array, due to miva's lack of
      'reflection' (ie: not easy to ask a structure "what branches do you
      have?" - at least not in 4.x and under).

      The error branch is the beauty of it, as it's let's you seperate error
      handling from the core functions. All they do is report the error,
      it's up to the calling function to decide what to do - which could
      include ignoring them. Of course, that does mean the burden shifts,
      but that can be handled with some generic error handling (which can
      vary by app, or by function call).

      In Merchant's db.mv code, there are a lot of functions that can't be
      used 'flexibly' because they cause the script to error out.
      Sometimes... "Category_Find_Code" is just a 'check', but if I need to
      check to see if a category code exists, I can't use that function due
      to it's hard coded error check.

      anyway, just thinking out loud, thought I'd share it.

      --
      Bill Guindon (aka aGorilla)



      Comment


        #4
        structured return values.



        On 5/16/05, Claudiu <[email protected]> wrote:
        > Hi Bill,
        >=20
        > Thanks for sharing your ideas and your research on the list. I hope other
        > ppl on that list ("community") will follow you example. There are just a =
        few
        > (but very valuable) who do that.. but in my opinion that's not enough..

        Funny thing is, it started out as a private email to a few people,
        then I decided to just throw it at the list for a wider audience.
        =20
        > Thanks again
        > Claudiu Bischoff
        >=20
        > -----Original Message-----
        > From: [email protected]
        > [mailto:[email protected]]On Behalf Of Bill Guindon
        > Sent: lundi 16 mai 2005 03:17
        > To: Miva Users
        > Subject: [meu] structured return values.
        >=20
        > had a thought earlier today related to structures as return values...
        >=20

        --=20
        Bill Guindon (aka aGorilla)


        Comment


          #5
          structured return values.



          On 5/16/05, Ivo Truxa <[email protected]> wrote:
          > Passing references or pointers in function arguments is a common techniqu=
          e
          > in many languages. It is of course much more efficient, especially at mor=
          e
          > complex arguments like structures or arrays, because it avoids creating t=
          he
          > temporary local variables and copying the original content to the stack (=
          or
          > standard operating memory in some cases) and then back again. References =
          not
          > only save all those extra CPU instructions needed for the creation of the
          > temporary variables, but it also avoids blocking additional memory.

          That was my take on it, much faster to pass a memory address than to
          recreate it on the fly. Seems that I got mixed results when testing
          it, but that was a couple years ago. I'll try to run some new
          benchmarks.
          =20
          > The disadvantage of breaking the intuitivity and readability is real, but
          > can be partially reduced by consistently following "Hungarian" naming
          > conventions of the functions: using a specific prefixes at those function=
          s,
          > so that the difference between a function with plain arguments and a
          > function using (and possibly modifying) references is apparent on the fir=
          st
          > look.

          Good point. Can take it a step further, and apply it to both, the
          function name, and the parameter names. If the function name has a
          prefix, it returns a structure (allowing for the error msg ideas I
          mentioned), if the paramater has a prefix, it's a sign that it may be
          modified in the function. Of course, you have to remember to use the
          naming on the parameters as that can't be enforced.

          > It certainly does not mean that references should be used everywhere inst=
          ead
          > of plain arguments. Understanding what happens on lower levels (OS / BIOS=
          /
          > Assembler / CPU ...) certainly helps very much when programming in high
          > level languages like Miva Script too, but generally you can consider that
          > passing arrays and structures as references is typically much more
          > efficient.

          One thing I'd like to compare is simple assign's (both left and right)
          - I'd imagine that 'simple' variables may have a slight speed
          advantage over structured variables. When I get the chance, I'll try
          some more benchmarking.

          > Ivo
          > http://mivo.truxoft.com
          >=20
          >=20
          > -----Original Message-----
          > From: Claudiu
          >=20
          > Hi Bill,
          >=20
          > Thanks for sharing your ideas and your research on the list. I hope other
          > ppl on that list ("community") will follow you example. There are just a =
          few
          > (but very valuable) who do that.. but in my opinion that's not enough..
          >=20
          > Thanks again
          > Claudiu Bischoff
          >=20
          > -----Original Message-----
          > From: [email protected]
          > [mailto:[email protected]]On Behalf Of Bill Guindon
          > Sent: lundi 16 mai 2005 03:17
          > To: Miva Users
          > Subject: [meu] structured return values.
          >=20
          > had a thought earlier today related to structures as return values...
          >=20
          > When I run into situations where I want a function to modify multiple
          > values, I tend to add those values as parameters, and pass them by VAR
          > then let the function modify them.
          >=20
          > This has a few side effects (some good, some bad)...
          >=20
          > not intuitive, you can't tell that the assign is happening simply by
          > looking at the function call.
          >=20
          > if the function doesn't return a value, you find yourself using
          > MvEVAL's, which just don't seem right.
          >=20
          > the function can still return a value, which I tend to use as the
          > 'l.ok' success/fail value.
          >=20
          > Now I've been using these approaches for some time. Part of the
          > reason for that, was a speed issue. It seemed that when you had very
          > large structures, the code ran faster (uncompiled) if you passed them
          > by var instead of passing them back as a return value. I'm not sure
          > whether this is still an issue or not, and should probably run some
          > tests to find out.
          >=20
          > Assuming speed is no longer an issue (compiled at least), I'm thinking
          > of changing it to structured return values. Dunno why it never
          > crossed my mind, and not sure how well it would work, but I'm thinking
          > if I use some standard naming convention, it could work.
          >=20
          > Can still return an 'ok' value, just as a branch:
          > l.ret:ok
          >=20
          > If it's ok, stuff any data you have on a data branch:
          > l.ret:data
          >=20
          > If you have errors, you can stick them on independent branches --
          > which allows you to track multiple errors, and/or respond to different
          > errors different ways.
          > l.ret:error:MvOpen
          > l.ret:error:EOF
          > l.ret:error:Fexists
          > l.ret:error:DataEntry
          >=20
          > It may also pay to make it a structured array, due to miva's lack of
          > 'reflection' (ie: not easy to ask a structure "what branches do you
          > have?" - at least not in 4.x and under).
          >=20
          > The error branch is the beauty of it, as it's let's you seperate error
          > handling from the core functions. All they do is report the error,
          > it's up to the calling function to decide what to do - which could
          > include ignoring them. Of course, that does mean the burden shifts,
          > but that can be handled with some generic error handling (which can
          > vary by app, or by function call).
          >=20
          > In Merchant's db.mv code, there are a lot of functions that can't be
          > used 'flexibly' because they cause the script to error out.
          > Sometimes... "Category_Find_Code" is just a 'check', but if I need to
          > check to see if a category code exists, I can't use that function due
          > to it's hard coded error check.
          >=20
          > anyway, just thinking out loud, thought I'd share it.
          >=20
          > --
          > Bill Guindon (aka aGorilla)
          >=20

          Comment


            #6
            structured return values.



            Hi aGorilla!

            Interesting subject... There are a few things that come to mind:

            1.) Regarding speed of variables by reference: A while ago, I ran some speed
            comparisons, and the result is actually quite impressing, somewhere in the
            range of 15-30% for x00000 loops in a compiled scripts (vars by reference
            being faster). Of course this greatly depends on the size of the variables
            and the computer/memory etc.

            This said, I still prefer the way that you described, meaning passing
            variables by value, simply for clarity sake. After all, even a 30%
            difference is not very much - after all we are talking about a few usecs per
            iteration at best.

            But there are two more important aspects for not using variables by
            reference too often, which has to do with what you wrote about error
            handling, as well as code reuse.


            2.) After trying lots of different approaches to find the silver bullet for
            an integrated / semi-automatic error catching & handling, I found that the
            following design pattern works quite well for me. To simplify implementing
            this in a consistent manner I use a precompiler that builds all the wrapping
            and writes the functions.

            On a high level, my scripts are built in form of subsystems, each performing
            some specific logical/programmatical tasks. This is similar to Merchant
            modules, but less restricted. Subsystems are always nested. Each subsystem
            has one interface, with a number of supporting functions / implementations.
            The source code is stored in a database, together with error codes and pre
            and postconditions, documentation as well as some profiling or debugging
            instructions. A subsystem is automatically entered when an interface is
            called.

            Subsystem
            InterfaceA(this,that)
            Precondition //checks that the input is correct
            ifError --->Errorhandling (correct error or exit)

            w_interfaceA(this,that) // main functionbody or dispatcher
            calls supporting functions or subsystems....
            ifError --->Errorhandling()

            Postcondition // ensures that the result is as expected
            ifError --->Errorhandling

            Both pre- and postconditions are optional. They are added at precompilation
            time to the source code and compiled, for performance reasons. The
            post-conditions might appear redundant, but they help when you look at the
            documentation to understand what is really expected from this particular
            program task.

            Each interface is nothing but a wrapper to the original entry point to
            perform task X. Ideally, this main function is a dispatcher that controls
            the flow within the subsystem (which happens in the supporting functions).
            There are a handful of validation functions in a library that are used to
            validate the pre- & postconditions, like for positive integer, arrays,
            structures, within a specific range, etc. I use a simple web-based IDE where
            I can add such a pre-condition in form of "validation.check_email(l.email)",
            the precompiler then generates the necessary MvDO to the validation routine.



            Interfaces can be called dynamically at runtime through simple tags (for
            example in templates or database values), supporting functions can't. This
            ensures that a user cannot bypass the errorhandling. It is however possible
            to bypass the wrappers if hardcoded in the source.



            In case of an error, a g.__runtime_errors:structure gets a few values that
            are defined in a database, so that the program knows what happened and what
            to do. Since this is a global variable, the error can propagate through the
            script until a function/interface is found that knows what to do with it to
            reset it. The neat thing is that after returning from a subsystem, the lower
            level interface can simply check for the presence of g.__runtime_errors to
            see if something went wrong.

            Since each wrapper stores the original input variables (locals), it can
            continue after an error without having these values been changed by a
            routine that went wrong. Obviously, changed database values and global
            variables must be handled separately.


            Since the subsystems are nested, the program/errorhandler can also calculate
            the nesting level (it maintains a simple log of all interface calls), and in
            case of an error that it can't fix automatically step back to an earlier
            (lower) level, and try to pick up the program flow from there, discarting
            what it unsuccessfully tried before. If instructed accordingly, a large
            program task that for instance calls a dozen other subsystems can be
            interrupted and jump back to the original lower level call, without passing
            through a dozen indidivual errorhandlers or the previous functions on the
            same program level.



            The main purpose of this approach is to isolate program parts as much as
            possible from each other, and to have a maximum of control through the
            interfaces, and not inside the program body. In a packaged version, the
            interfaces/wrappers are actually open-source, while the supporting functions
            and implementations can remain closed. So clients have full control about
            the program flow, errorhandling and thanks to the pre and post conditions
            can see what is required/expected, but I can still keep some of my secrets
            or protect program elements from changes. The wrapper also allows to switch
            on/off very fine-grained profiling and debugging settings on a subsystem
            level.



            Isolating those subsystems also ensures that it is much easier to reuse them
            in different programs, since they are pretty much independent and get their
            data through the interfaces and not through global or referenced variables.
            Referenced vars should really only be used within the same subsystem in
            calls between the supporting functions.


            One thing that's really difficult for me is that it is always very tempting
            to write errorhandlers into the main program body (where the error initially
            occurs ), instead of relying on the interfaces to handle them in a more
            organised/structured/global way. I still tend to make functions too
            intelligent, which always messes up the code and makes them hard to read and
            debug. The tricky thing is find a healthy balance between making a program
            too fine-grained or to let a function do too much. Oh well...


            Markus














            -----Original Message-----
            From: [email protected]
            [mailto:[email protected]] On Behalf Of Bill Guindon
            Sent: Montag, 16. Mai 2005 03:17
            To: Miva Users
            Subject: [meu] structured return values.

            had a thought earlier today related to structures as return values...

            When I run into situations where I want a function to modify multiple
            values, I tend to add those values as parameters, and pass them by VAR then
            let the function modify them.

            This has a few side effects (some good, some bad)...

            not intuitive, you can't tell that the assign is happening simply by looking
            at the function call.

            if the function doesn't return a value, you find yourself using MvEVAL's,
            which just don't seem right.

            the function can still return a value, which I tend to use as the 'l.ok'
            success/fail value.

            Now I've been using these approaches for some time. Part of the reason for
            that, was a speed issue. It seemed that when you had very large structures,
            the code ran faster (uncompiled) if you passed them by var instead of
            passing them back as a return value. I'm not sure whether this is still an
            issue or not, and should probably run some tests to find out.

            Assuming speed is no longer an issue (compiled at least), I'm thinking of
            changing it to structured return values. Dunno why it never crossed my
            mind, and not sure how well it would work, but I'm thinking if I use some
            standard naming convention, it could work.

            Can still return an 'ok' value, just as a branch:
            l.ret:ok

            If it's ok, stuff any data you have on a data branch:
            l.ret:data

            If you have errors, you can stick them on independent branches -- which
            allows you to track multiple errors, and/or respond to different errors
            different ways.
            l.ret:error:MvOpen
            l.ret:error:EOF
            l.ret:error:Fexists
            l.ret:error:DataEntry

            It may also pay to make it a structured array, due to miva's lack of
            'reflection' (ie: not easy to ask a structure "what branches do you have?" -
            at least not in 4.x and under).

            The error branch is the beauty of it, as it's let's you seperate error
            handling from the core functions. All they do is report the error, it's up
            to the calling function to decide what to do - which could include ignoring
            them. Of course, that does mean the burden shifts, but that can be handled
            with some generic error handling (which can vary by app, or by function
            call).

            In Merchant's db.mv code, there are a lot of functions that can't be used
            'flexibly' because they cause the script to error out.
            Sometimes... "Category_Find_Code" is just a 'check', but if I need to check
            to see if a category code exists, I can't use that function due to it's hard
            coded error check.

            anyway, just thinking out loud, thought I'd share it.

            --
            Bill Guindon (aka aGorilla)


            Comment

            Working...
            X