Feature request: more memory clearing

[DavidW]

Could we have something like

Code: [Select]

CLEAR varwhich would just unset the 'var' variable so that VARIABLE_IS_SET returns false for it (etc.)? Like CLEAR_MEMORY, but selectively.

Could we also have

Code: [Select]

CLEAR_ARRAY_ENTRIES myarraywhich would be equivalent to

Code: [Select]

ACTION_PHP_EACH myarray AS k=>v BEGIN
    CLEAR "myarray_%k%"
END
ACTION_CLEAR_ARRAY myarray
and so wouldn't be essential if we had CLEAR, since I could always use a macro, but would be convenient.

[Wisp]

I seem to recall rejecting this request in the past, because I think this is why we have variable scopes and that CLEAR_MEMORY is a relic from the before-time, but presumably you wouldn't be requesting it if functions et al. were sufficient, so what am I missing?

[DavidW]

My main use case is being able to define a guaranteed-empty array inside a function. As a stylized example:

Code: [Select]

DEFINE_PATCH_FUNCTION fn BEGIN
      CLEAR_ARRAY my_array
      $my_array("Minsc")=1
      $my_array("Jaheira")=1
     PATCH_FOR_EACH npc IN Minsc Jaheira Sarevok BEGIN
         PATCH_IF VARIABLE_IS_SET $my_array("%npc%") BEGIN
             PATCH_PRINT "%npc% is in my_array"
         END
     END
END

The intention is that Minsc and Jaheira are returned as in 'my_array', but Sarevok isn't.
But if I call the function in the scope of a previous declaration of $my_array("Sarevok")=1, or even my_array_Sarevok=1, then of course the function doesn't do what it's supposed to do.

I guess what's going on is that arrays in WEIDU aren't really arrays, just dictionaries into organized collections of variables, and CLEAR_ARRAY just clears the dictionary.

It can be worked around without too much trouble, so don't worry if it's awkward to do.

[DavidW]

Here's a less artificial example from my local function library:

Code: [Select]

DEFINE_DIMORPHIC_FUNCTION 2da_inject_array
INT_VAR silent=0//boolean
STR_VAR array=""//array
array_in=""//array
column=""
RET_ARRAY array
BEGIN
  OUTER_PATCH "" BEGIN
CLEAR_ARRAY rows
CLEAR_ARRAY columns
TO_UPPER column
LPF 2da_extract_rows_columns STR_VAR array RET_ARRAY rows columns END
LPF array_map STR_VAR array=array_in keymap=toupper RET_ARRAY array_in=array END
PATCH_IF !VARIABLE_IS_SET $columns("%column%") BEGIN
PATCH_IF !silent BEGIN
PATCH_WARN "2da_inject_array: column %column% is absent from 2d array %array%"
END
END ELSE BEGIN
PHP_EACH rows AS row=>discard BEGIN
PHP_EACH columns AS col=>discard BEGIN
PATCH_IF VARIABLE_IS_SET $"%array_in%"("%row%") && "%col%" STR_EQ "%column%" BEGIN
SPRINT $array("%row%" "%col%") $"%array_in%"("%row%")
END ELSE BEGIN
SPRINT $array("%row%" "%col%") $"%array%"("%row%" "%col%")
END
END
END
END
  END
END

Again, if the 'columns' array happens to have been set in scope, there could be false negatives. I could try to use some kind of 'reserved' convention for the names of arrays in functions, but that rather obviates the point of functions in scope control. And I could write an array_contains function, but it would have to cycle through the whole array, which is a bit inelegant and theoretically a performance hit.

EDIT: actually, come to think of it in this particular case I need to cycle through 'columns' anyway, so I can just move the check. But the general point remains.

[_Luke_]

I seem to recall rejecting this request in the past, because I think this is why we have variable scopes and that CLEAR_MEMORY is a relic from the before-time, but presumably you wouldn't be requesting it if functions et al. were sufficient, so what am I missing?

@DavidW
It is me that asked for that in the past (see here). As you can see, I also asked for CLEAR_INLINE (like CLEAR_INLINED, but selectively) and CLEAR_CODE (like CLEAR_CODES, but selectively), but that's probably unnecessary unlike CLEAR_VARIABLE...

You should try using WITH_SCOPE / PATCH_WITH_SCOPE and see if it helps. So for instance, turn this

Code: [Select]

ACTION_CLEAR_ARRAY "patch_data"
ACTION_DEFINE_ASSOCIATIVE_ARRAY "patch_data" BEGIN
  // something
END
LAF "make_item" STR_VAR "edits" = "patch_data" END
into this

Code: [Select]

ACTION_CLEAR_ARRAY "patch_data" // Do this just once at the beginning of a component, just in case
WITH_SCOPE BEGIN
  ACTION_DEFINE_ASSOCIATIVE_ARRAY "patch_data" BEGIN
    // something
  END
  LAF "make_item" STR_VAR "edits" = "patch_data" END
END
In so doing, you're 100% sure that the array "patch_data" is properly cleared.

However, I agree with you that there are cases where having a CLEAR_VARIABLE command would be certainly handy...

[DavidW]

I don't think WITH_SCOPE solves my particular issue. It doesn't block ambient variables.

At an abstracted level, the issue looks like this. Functions need to be able to use internal variables that can be called whatever you like without worrying about how they interact with the external environment: they only interact with that environment through their input and return variables. That works fine for WEIDU with ordinary variables, but not with arrays, because there's no way to define an entirely new array in WEIDU short of CLEAR_MEMORY. You can define a new array dictionary with CLEAR_ARRAY but there's (I think) no way to clear the individual array elements.

(The robust answer would just be that WEIDU arrays are dictionaries and you shouldn't kid yourself that they're proper arrays in the sense that other languages have them. But they are so close to being like proper arrays!)

[DavidW]

OK, reflecting further on this I think I retract the request. I can see that it's subtle - and possibly difficult - to implement array clearing given how WEIDU arrays work. And testing a quick array_contains function, I find that WEIDU can check if a string is a key in a 10,000 element array in 0.08 seconds, so the concern about performance is overrated.

[_Luke_]

OK, reflecting further on this I think I retract the request. I can see that it's subtle - and possibly difficult - to implement array clearing given how WEIDU arrays work. And testing a quick array_contains function, I find that WEIDU can check if a string is a key in a 10,000 element array in 0.08 seconds, so the concern about performance is overrated.

OK, so how can you clear array entries then?

Based on what you said here, the following code

Code: [Select]

DEFINE_DIMORPHIC_FUNCTION "clear_array_entries"
RET_ARRAY "empty_array"
BEGIN
OUTER_SET $"empty_array"("whocares") = 1
END

LAF "clear_array_entries" RET_ARRAY "my_array" = "empty_array" ENDshould turn an array into an "empty" one (it's not really empty though since it contains a dummy entry to bypass that install-time error).

However, it's not working.
That is, if I replace CLEAR_ARRAY "my_array" with LAF "clear_array_entries" RET_ARRAY "my_array" = "empty_array" END in your function fn and $"my_array"("Sarevok") happens to be defined, then fn will still print Sarevok is in my_array.

What am I missing...?

[DavidW]

The array is empty, but the underlying data set isn't empty.

A little terminology might help: given a string (say, ‘myarray’), then a ‘struct’ (my term) is the collection of all variables with that string as prefix: the myarrray struct might contain myarray_0, myarray_1, myarray_2_fighter, myarray_Tiax, etc., etc. The myarray array is a dictionary into some, but not necessarily all, of the myarray struct. Using ‘myarray_Tiax=1’ adds to the struct but not to the array; using ‘$myarray(“Tiax”)=1’, or the (ACTION_)DEFINE_ASSOCIATIVE_ARRAY command, adds to both. And if you (ACTION_)CLEAR_ARRAY myarray, you wipe the dictionary but leave the struct alone.

As to how you can clear array entries: you can't. You can clear the array (the dictionary) but not the underlying struct (short of CLEAR_MEMORY). On reflection I don't think that's a problem: this function works fine.

Code: [Select]

DEFINE_DIMORPHIC_FUNCTION array_contains
STR_VAR array=""//array
key=""
val=""
RET value
BEGIN
OUTER_SET value=0
ACTION_PHP_EACH "%array%" AS k1=>v1 BEGIN
ACTION_IF ("%key%" STRING_EQUAL "%k1%" || "%key%" STR_EQ "") && ("%val%" STRING_EQUAL "%v1%" || "%val%" STR_EQ "") BEGIN
OUTER_SET value=1
END
END
END


[Wisp]

So is there still a feature request?

[_Luke_]

So is there still a feature request?

Probably not, since the following code is now working as intended:

Code: [Select]

DEFINE_PATCH_FUNCTION "fn" BEGIN
  CLEAR_ARRAY "my_array"
  SET $"my_array"("Minsc") = 1
  SET $"my_array"("Jaheira") = 1
  PATCH_FOR_EACH "npc" IN "Minsc" "Jaheira" "Sarevok" BEGIN
    LPF "array_contains"
    STR_VAR
      "array" = "my_array"
      "key" = "%npc%"
    RET
      "value"
    END
    PATCH_IF "%value%" BEGIN
      PATCH_PRINT "%npc% is in my_array"
    END
  END
END

// Main

OUTER_SET $"my_array"("Sarevok") = 1

OUTER_PATCH "" BEGIN
    LPF fn END // Will no longer print ~Sarevok is in my_array~
END


Maybe you could add DavidW's array_contains function to WeiDU's repertoire...

@DavidW
Related: what about "killing" the ACTION_PHP_EACH loop after finding what you were looking for?

Code: [Select]

DEFINE_DIMORPHIC_FUNCTION array_contains
STR_VAR array=""//array
key=""
val=""
RET value
BEGIN
OUTER_SET value=0
ACTION_PHP_EACH "%array%" AS k1=>v1 BEGIN
    ACTION_IF !("%value%") BEGIN
    ACTION_IF ("%key%" STRING_EQUAL "%k1%" || "%key%" STR_EQ "") && ("%val%" STRING_EQUAL "%v1%" || "%val%" STR_EQ "") BEGIN
    OUTER_SET value=1
    END
  END
END
END

[DavidW]

So is there still a feature request?

I don't think so.

[_Luke_]

So is there still a feature request?

On reflection, yes, there is.

What about multi-dimensional arrays? The aforementioned function would not work as intended in such cases...
As a completely artificial example:

Code: [Select]

ACTION_DEFINE_ASSOCIATIVE_ARRAY "my_array" BEGIN
1 , 2 => 3
END

LAF "array_contains"
STR_VAR
"array" = "my_array"
"key" = "1"
RET
"value" // will return 1 (true); should return 0 (false) since the key is "1_2" (not just "1")
END

I don't think it is possible to tweak array_contains so that it can take into account multi-dimensional arrays, specifically because VARIABLE_IS_SET is unreliable when dealing with arrays...

What do you think...? Am I missing something...?

[DavidW]

No, I think that's pretty much right. (Though it's not that the key is '1_2', it's that there are 2 keys, 1 and 2.)

I think rather than memory clearing, it would be better (if possible) just to have some way to detect if an array element is set. say

ARRAY_IS_SET string (string list)

so that ARRAY_IS_SET my_array (1) would return false in Luke's example, but ARRAY_IS_SET my_array (1 3) would return true.

[_Luke_]

(Though it's not that the key is '1_2', it's that there are 2 keys, 1 and 2.)

Yeah, you are right... I wrote "1_2" because I thought about "%my_array_1_2%" (as an alternative to $"my_array"("1" "2")...)

I think rather than memory clearing, it would be better (if possible) just to have some way to detect if an array element is set. say

ARRAY_IS_SET string (string list)

so that ARRAY_IS_SET my_array (1) would return false in Luke's example, but ARRAY_IS_SET my_array (1 2) would return true.

Sounds good.

[Wisp]

So if I'm understanding this correctly, this proposed ARRAY_IS_SET would function like VARIABLE_IS_SET except it also checks that the variable is a member of the given array (and some syntactic sugar for arrays)? I'll see what I can do.

[_Luke_]

So if I'm understanding this correctly, this proposed ARRAY_IS_SET would function like VARIABLE_IS_SET except it also checks that the variable is a member of the given array?

Basically, yes.