Helpful ReplyHow to remove duplicate midi notes with Undupe CAL?

Author
twelvetone
Max Output Level: -89 dBFS
  • Total Posts : 70
  • Joined: 2015/08/19 16:47:20
  • Status: offline
2015/08/22 20:00:22 (permalink)

How to remove duplicate midi notes with Undupe CAL?

After editing a midi bassline, I have discovered double overlaying notes.
Done some searching and the solution apparently is the "undupe" cal script.
 
I cannot get it to work.
 
At the start, it asks for "Window for time identity", which is 0 by default.
No matter what I enter, it returns with "deleted 0 duplicate events".
 
It does not even find a double pair of notes I put in manually.

Anyone know how it is supposed to work?
 
I have Sonar X3e.
post edited by twelvetone - 2015/08/22 20:09:13
#1
Notecrusher
Max Output Level: -79 dBFS
  • Total Posts : 579
  • Joined: 2004/02/17 00:32:14
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/23 05:48:28 (permalink)
Yeah that CAL is not working for me either. A lot of old CAL scripts don't work anymore due to the changes to the s/w over the years.
#2
twelvetone
Max Output Level: -89 dBFS
  • Total Posts : 70
  • Joined: 2015/08/19 16:47:20
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/23 15:44:55 (permalink)
This is getting disheartening.
 
So often when I try out something in Sonar, it's outdated, buggy or doesn't work properly anymore.
 
Are new features being piled on top of a crumbling core?
 
I found a 101 page document from 2010 called Cakewalk Application Language Programming Guide,
so I'm teaching myself CAL now so I can get to the bottom of this problem.
post edited by twelvetone - 2015/08/23 16:22:16
#3
Notecrusher
Max Output Level: -79 dBFS
  • Total Posts : 579
  • Joined: 2004/02/17 00:32:14
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/23 17:21:38 (permalink)
CAL is very old and is not supported anymore. There's plenty of CAL scripts that still work but you can't be guaranteed that any particular CAL will work because it is a deprecated feature. I suggest up-voting this feature request:
http://forum.cakewalk.com/Replace-CAL-With-Python-m3098713.aspx
 
 
post edited by Notecrusher - 2015/08/23 17:39:43
#4
icontakt
Max Output Level: -32.5 dBFS
  • Total Posts : 4266
  • Joined: 2012/03/04 08:18:02
  • Location: Tokyo
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/23 21:51:25 (permalink)
I can also confirm that Undupe no longer works. I don't know when it got broken, but I was planning to use it on many projects in the near future, so this is very bad news for me.
 
twelvetone
This is getting disheartening.
 
So often when I try out something in Sonar, it's outdated, buggy or doesn't work properly anymore.
 
Are new features being piled on top of a crumbling core?


 
I think so. There are a number of bugs that haven't been fixed for more than 1-2 years after they were found and reported. CW gave higher priority to redesigning the start page, which nobody really asked for, over fixing a bug that 40+ users wanted fixed.
 
I often wonder if it's the same in the other old daws like PT, DP, and Cubase.

Tak T.
 
Primary Laptop: Core i7-4710MQ CPU, 16GB RAM, 7200RPM HDD, Windows 7 Home Premium OS (Japanese) x64 SP1
Secondary Laptop: Core2 Duo CPU, 8GB RAM, 7200RPM HDD, Windows 7 Professional OS (Japanese) x64 SP1
Audio Interface: iD14 (ASIO)
Keyboard Controller/MIDI Interface: A-800PRO
DAW: SONAR Platinum x64 (latest update installed)
#5
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 01:47:49 (permalink)
The script could be simplified but it appears to work OK.
 
The script works on freshly selected clips (IOW, if the script is aborted make another selection before running the script again). If no clips are selected, nothing will change.
 
The user input is in Ticks. The value is divided by 2 then added to and subtracted from the last note start time to determine of the last note overlaps the current note. If it does overlap and the note key values are the same, the script deletes the current note.
 
#6
Notecrusher
Max Output Level: -79 dBFS
  • Total Posts : 579
  • Joined: 2004/02/17 00:32:14
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 02:37:04 (permalink)
It does not work in Splat.
#7
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 02:45:05 (permalink)
I tested in SONAR Platinum before posting.
#8
icontakt
Max Output Level: -32.5 dBFS
  • Total Posts : 4266
  • Joined: 2012/03/04 08:18:02
  • Location: Tokyo
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 02:57:53 (permalink)
Maybe it's not working properly? (Just tried it in the Normal project template)
 


Tak T.
 
Primary Laptop: Core i7-4710MQ CPU, 16GB RAM, 7200RPM HDD, Windows 7 Home Premium OS (Japanese) x64 SP1
Secondary Laptop: Core2 Duo CPU, 8GB RAM, 7200RPM HDD, Windows 7 Professional OS (Japanese) x64 SP1
Audio Interface: iD14 (ASIO)
Keyboard Controller/MIDI Interface: A-800PRO
DAW: SONAR Platinum x64 (latest update installed)
#9
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 03:13:28 (permalink)
Did you try changing the time window?
#10
icontakt
Max Output Level: -32.5 dBFS
  • Total Posts : 4266
  • Joined: 2012/03/04 08:18:02
  • Location: Tokyo
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 03:13:51 (permalink) ☄ Helpfulby taccess 2017/09/06 01:58:44
I just did some more testing and it seems that Undupe ignores the first set of overlapped notes in the selection when no other events exist before them.
In other words, it's better to merge all MIDI clips in the track before using this script.

Tak T.
 
Primary Laptop: Core i7-4710MQ CPU, 16GB RAM, 7200RPM HDD, Windows 7 Home Premium OS (Japanese) x64 SP1
Secondary Laptop: Core2 Duo CPU, 8GB RAM, 7200RPM HDD, Windows 7 Professional OS (Japanese) x64 SP1
Audio Interface: iD14 (ASIO)
Keyboard Controller/MIDI Interface: A-800PRO
DAW: SONAR Platinum x64 (latest update installed)
#11
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 03:15:00 (permalink)
As I said, the script can be simplified. The time window setting is not particularly intuitive either.
#12
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 03:20:53 (permalink)
icontakt
In other words, it's better to merge all MIDI clips in the track before using this script.

Keep in mind CAL predates lanes and layers. Only MIDI tracks existed when CAL was developed.
#13
icontakt
Max Output Level: -32.5 dBFS
  • Total Posts : 4266
  • Joined: 2012/03/04 08:18:02
  • Location: Tokyo
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 03:31:11 (permalink)
scook
Did you try changing the time window?

 
Just did it, and got the same results.
 
scook
As I said, the script can be simplified. The time window setting is not particularly intuitive either.

 
I haven't looked into the script, but if it's not easy for me to fix I'll just merge all clips before unduping.
 
scook
icontakt
In other words, it's better to merge all MIDI clips in the track before using this script.

Keep in mind CAL predates lanes and layers. Only MIDI tracks existed when CAL was developed.


 
Yes, but I only used one lane in the test above.
 
 
Anyway, I thought Undupe got broken sometime this year (because I only tried it on one set of duplicated notes when I posted #5). I'm glad it didn't. I'm OK with the workaround.

Tak T.
 
Primary Laptop: Core i7-4710MQ CPU, 16GB RAM, 7200RPM HDD, Windows 7 Home Premium OS (Japanese) x64 SP1
Secondary Laptop: Core2 Duo CPU, 8GB RAM, 7200RPM HDD, Windows 7 Professional OS (Japanese) x64 SP1
Audio Interface: iD14 (ASIO)
Keyboard Controller/MIDI Interface: A-800PRO
DAW: SONAR Platinum x64 (latest update installed)
#14
twelvetone
Max Output Level: -89 dBFS
  • Total Posts : 70
  • Joined: 2015/08/19 16:47:20
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 07:17:58 (permalink)
I am working through the Cakewalk Application Language Programming Guide.pdf
I made some test scripts which successfully change Now, track titles, get the event kind, etc.
So fundamentally, CAL is there and still works.
 
The scripts seem to be divided into 2 types:
1) Ones with author details inside. These have filenames that conform to the DOS 8.3 all-caps format and appear to to originate from a competition held in 1991. Each author has an ID and seems keen to publish personal details. All of these have a 3-part structure comprising 3 do-blocks which, in some file comments, are identified as Prolog, Body and Epilog. UNDUPE.CAL is one of these, the attribution is however a bit vague. It only states "Maybe Miguel Ratton".
 
2) Ones with no author in the comments. These have mixed-case long names. None have originator or  author info.
These are also structurally different in that they have a single do block and most have looping statements such as while and, significantly, forEachEvent.
 
There is one from this group that stands out: Split Note to Tracks.cal in that it has in the comments:
Courtesy Red Nile Productions 1993
Modified Greg Hendershott at user's request 12/10/93
 
There is also a script called need20.cal which is called up by many other scripts. All it does is check whether the CAL engine is version 2.0 or higher. (X3 is 100 = v10.0)
 
I am not deep enough into CAL yet but my guess is that the older scripts conform to this 3-block Prolog-Body-Epilog idiom which carries with it an implied looping and that this is now superseded by explicit looping statements, which may explain why icontakt finds only one note is being deleted from a selected range in UNDUPE.CAL. (Although I am not getting it to delete ANY yet...)
 
Having said that, I am finding that I can play with the 3-block idiom and it does seem to work to some degree.
 
I have found no mention of the 3-block idiom in the document so far, so this is all speculation at this point.
 
post edited by twelvetone - 2015/08/24 07:43:31
#15
twelvetone
Max Output Level: -89 dBFS
  • Total Posts : 70
  • Joined: 2015/08/19 16:47:20
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 07:26:20 (permalink)
It would be useful to know whether track lanes and layers have been assigned identifiers. Tricky, because scripts could exist using these names already.
 
CAL is pretty powerful, just as UNDUPE.CAL demonstrates.
 
It would be nice to have a documentation. It needn't be a LISP programming course, just document the reserved and key words, and give some sample code.
#16
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 12:16:29 (permalink)
I downloaded the CAL Editor years ago not for editor but for the "CAL Reference.chm"
No worked examples but a nice concise reference.
#17
twelvetone
Max Output Level: -89 dBFS
  • Total Posts : 70
  • Joined: 2015/08/19 16:47:20
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 13:06:21 (permalink)
Curious. The help files open, but if you click on anything in the left tree control, the right side text window remains blank.
 
OK, I now tried on Windows XP. They work. But not on Windows 8.1, evidently.
 
scook, as ever, thanks so much for the support. Got something to do this weekend now.
#18
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 13:08:30 (permalink)
I am still on Win7
#19
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 13:24:39 (permalink)
Here is a different take on removing duplicate notes. Did not test it much. Functionally similar to UNDUPE.CAL
; rmdupes.cal
; scook 2015

; set up variables and get user input
(do
 (dword lastTime 0)
 (int lastKey 0)
 (int nDeleted 0)
 (int timeWindow 0)

; If a note starts this number of ticks before or after the current note
; and is the same pitch, it is considered a duplicate.
 (getInt timeWindow "Time (in ticks) to determine dupes:" 0 32767))

; Loop through the selected events. NOTE: the current implementation
; of SONAR (2015) does not reset the start if the script is aborted. IOW,
; if the script is aborted, reselect the values to start over.
(do
 ; Only process notes
 (if (== Event.Kind NOTE)
     (do
      ; Check if the note is the same key as the previous note and
      ; fall within the time range provided. If it meets these criteria
      ; delete the note otherwise remember the note time and key
      ; for the next comparison.
      (if (&& (== Note.Key lastKey)
          (&& (>= Event.Time (- lastTime timeWindow))
          (<= Event.Time (+ lastTime timeWindow))))
      (do
       (delete)
       (++ nDeleted))
    (do
     (= lastTime Event.Time)
     (= lastKey Note.Key))))))
; report
(do
 (pause "Deleted " nDeleted " duplicate notes."))
#20
Notecrusher
Max Output Level: -79 dBFS
  • Total Posts : 579
  • Joined: 2004/02/17 00:32:14
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 17:20:20 (permalink)
Yeah it's just the first duplicated pair undupe misses for some reason.
#21
msorrels
Max Output Level: -70 dBFS
  • Total Posts : 1025
  • Joined: 2003/11/08 02:04:59
  • Location: Pittsburgh, PA
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/24 17:52:48 (permalink)
Don't you need to process each key/note value lane and remove duplicates along it rather than just process each selected event?  Doing just the events seems like you will miss a bunch of duplicates since you only remember one last note value.
#22
icontakt
Max Output Level: -32.5 dBFS
  • Total Posts : 4266
  • Joined: 2012/03/04 08:18:02
  • Location: Tokyo
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/25 09:03:50 (permalink)
scook
Here is a different take on removing duplicate notes. Did not test it much. Functionally similar to UNDUPE.CAL

 
Thanks for that, scook. But.....am I right in understanding that this CAL (I tried both the original Undupe and your rmdupes) only removes duplicate notes where no other note events exist (vertically)?
If so, it's not usable for, say, hard quantized drum tracks, is it?
 


Tak T.
 
Primary Laptop: Core i7-4710MQ CPU, 16GB RAM, 7200RPM HDD, Windows 7 Home Premium OS (Japanese) x64 SP1
Secondary Laptop: Core2 Duo CPU, 8GB RAM, 7200RPM HDD, Windows 7 Professional OS (Japanese) x64 SP1
Audio Interface: iD14 (ASIO)
Keyboard Controller/MIDI Interface: A-800PRO
DAW: SONAR Platinum x64 (latest update installed)
#23
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/25 11:08:57 (permalink)
That is correct. Neither script handles that case. I made no attempt to improve the performance of the original script. For a script to handle duplicates when notes occur in the same time window at different note values, the script would have to loop through the selection for each MIDI note value processing each separately.
#24
Notecrusher
Max Output Level: -79 dBFS
  • Total Posts : 579
  • Joined: 2004/02/17 00:32:14
  • Status: offline
Re: How to remove duplicate midi notes with Undupe CAL? 2015/08/25 18:45:46 (permalink)
icontakt
this CAL (I tried both the original Undupe and your rmdupes) only removes duplicate notes where no other note events exist (vertically)?
If so, it's not usable for, say, hard quantized drum tracks, is it?

 
no, nor for hard quantized chords.
#25
Jump to:
© 2025 APG vNext Commercial Version 5.1