You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 10:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Sat Mar 21, 2026 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3846405995023,8cyclictest3935199-21sh23:25:013
3846405994918,8cyclictest3912834-21sendmail22:20:003
3846405994732,11cyclictest391ktimers/322:40:103
3846396994738,6cyclictest3927694-21unixbench_multi23:00:231
3846396994738,6cyclictest3927694-21unixbench_multi23:00:231
3846405994617,4cyclictest0-21swapper/320:20:253
3846401994639,4cyclictest3912820-21cut22:20:012
3846392994640,4cyclictest3898814-21kworker/u8:1+events_unbound21:56:520
3846392994639,5cyclictest3831847-21kworker/u8:0+events_unbound19:16:300
3846396994534,5cyclictest3870846-21kworker/u8:0+events_unbound21:11:451
3846392994536,7cyclictest121ktimers/022:00:130
3846392994536,7cyclictest121ktimers/022:00:130
3846396994337,4cyclictest3870846-21kworker/u8:0+events_unbound20:56:421
3846392994337,4cyclictest3957826-21kworker/u8:1+events_unbound00:34:140
3846392994335,4cyclictest3947164-21modprobe23:59:590
3846392994335,4cyclictest3947164-21modprobe23:59:590
3846392994335,4cyclictest3942034-21awk23:45:000
3846405994228,11cyclictest391ktimers/320:45:123
3846401994235,4cyclictest3905985-21cat22:00:012
3846392994224,6cyclictest131rcu_preempt22:20:160
3846405994113,6cyclictest3848168-21latency_hist19:15:013
3846392994114,9cyclictest11-21ksoftirqd/020:20:010
3846405994030,7cyclictest3931311-21grep23:10:223
3846401994034,4cyclictest3909401-21sh22:10:002
3846401994034,4cyclictest3909401-21sh22:10:002
3846396994032,5cyclictest3881685-21munin-plugin-st20:50:021
3846392994028,9cyclictest3960985-21cron00:40:010
3846401993932,4cyclictest3875917-21grep20:30:252
3846396993931,5cyclictest3921355-21idleruntime-cro22:45:001
3846392993932,5cyclictest3870846-21kworker/u8:0+events_unbound20:51:410
3846392993931,5cyclictest3892113-21cron21:20:010
3846392993927,7cyclictest0-21swapper/021:36:460
3846392993910,21cyclictest131rcu_preempt22:40:220
3846401993830,5cyclictest3917918-21cron22:35:002
3846401993826,7cyclictest0-21swapper/200:05:262
3846392993826,5cyclictest3850567-21kworker/u8:3-events_freezable_power_19:35:030
3846392993820,4cyclictest131rcu_preempt21:21:360
3846405993727,7cyclictest3858179-21cat19:40:253
3846405993726,7cyclictest3893404-21grep21:20:263
3846401993730,5cyclictest3878051-21munin-plugin-st20:40:012
3846401993728,5cyclictest0-21swapper/223:20:002
3846401993728,5cyclictest0-21swapper/221:15:002
3846396993731,4cyclictest3867538-21kworker/u8:2+events_unbound20:16:371
3846392993730,3cyclictest0-21swapper/021:50:010
3846392993719,4cyclictest0-21swapper/019:53:050
3846392993719,4cyclictest0-21swapper/019:53:050
3846392993718,4cyclictest131rcu_preempt19:30:170
3846392993716,9cyclictest131rcu_preempt21:01:300
3846405993628,5cyclictest3888510-21munin-plugin-st21:10:003
3846405993627,4cyclictest0-21swapper/321:55:013
3846405993626,6cyclictest3868885-21grep20:10:243
3846405993626,6cyclictest0-21swapper/323:20:003
3846401993626,6cyclictest0-21swapper/223:25:002
3846396993628,4cyclictest0-21swapper/120:26:361
3846396993618,5cyclictest131rcu_preempt00:10:111
3846396993615,5cyclictest131rcu_preempt19:35:121
3846392993625,4cyclictest131rcu_preempt23:29:040
3846392993619,4cyclictest131rcu_preempt23:07:140
3846392993618,4cyclictest131rcu_preempt22:14:090
3846405993525,5cyclictest0-21swapper/300:25:153
3846401993527,4cyclictest0-21swapper/223:40:002
3846401993525,6cyclictest0-21swapper/222:45:012
3846401993524,7cyclictest0-21swapper/220:05:272
3846396993527,4cyclictest0-21swapper/122:10:001
3846396993527,4cyclictest0-21swapper/122:10:001
3846396993527,4cyclictest0-21swapper/119:55:011
3846396993527,4cyclictest0-21swapper/119:55:011
3846396993524,8cyclictest3926018-21grep22:55:251
3846396993524,8cyclictest3926018-21grep22:55:251
3846392993527,4cyclictest0-21swapper/020:45:250
3846392993521,6cyclictest3841116-21kworker/u8:1+events_unbound19:20:410
3846392993518,4cyclictest131rcu_preempt19:11:150
3846392993517,4cyclictest0-21swapper/020:40:480
3846392993516,3cyclictest0-21swapper/020:29:260
3846392993516,10cyclictest131rcu_preempt22:08:440
3846392993516,10cyclictest131rcu_preempt22:08:440
3846392993516,10cyclictest131rcu_preempt20:11:410
3846392993514,7cyclictest11-21ksoftirqd/020:05:260
3846405993423,7cyclictest391ktimers/323:30:123
3846401993426,4cyclictest3947308-21latency_hist00:00:012
3846401993426,4cyclictest3947308-21latency_hist00:00:002
3846401993425,6cyclictest0-21swapper/220:55:252
3846401993424,6cyclictest0-21swapper/200:25:242
3846401993423,6cyclictest0-21swapper/221:15:242
3846396993427,5cyclictest3893479-21kworker/u8:3+events_unbound21:39:431
3846396993426,4cyclictest0-21swapper/121:30:241
3846396993426,4cyclictest0-21swapper/120:50:161
3846396993424,6cyclictest3909846-21fschecks_time22:10:161
3846392993421,9cyclictest3917524-21grep22:30:250
3846392993421,5cyclictest131rcu_preempt22:48:080
3846392993417,5cyclictest131rcu_preempt23:00:130
3846392993417,5cyclictest131rcu_preempt23:00:130
3846392993417,4cyclictest131rcu_preempt22:37:010
3846392993416,10cyclictest131rcu_preempt22:29:530
3846392993416,10cyclictest131rcu_preempt22:16:240
3846392993416,10cyclictest131rcu_preempt21:28:530
3846392993414,5cyclictest131rcu_preempt00:15:240
3846405993325,4cyclictest0-21swapper/323:50:163
3846405993325,4cyclictest0-21swapper/323:50:153
3846405993322,8cyclictest3850338-21grep19:20:013
3846405993322,7cyclictest3882950-21grep20:50:243
3846401993326,4cyclictest3871035-21sendmail20:20:012
3846401993324,4cyclictest0-21swapper/200:35:242
3846401993323,6cyclictest0-21swapper/219:55:002
3846401993323,6cyclictest0-21swapper/219:55:002
3846396993324,6cyclictest3850767-21fschecks_time19:20:151
3846396993323,6cyclictest0-21swapper/123:35:261
3846392993321,4cyclictest131rcu_preempt21:10:550
3846392993320,5cyclictest131rcu_preempt20:34:230
3846392993313,11cyclictest0-21swapper/000:21:070
3846392993312,11cyclictest131rcu_preempt23:10:200
3846392993310,5cyclictest131rcu_preempt23:50:170
3846392993310,5cyclictest131rcu_preempt23:50:160
3846405993222,7cyclictest3936433-21grep23:25:263
3846401993227,3cyclictest3953908-21sh00:17:012
3846401993223,6cyclictest0-21swapper/223:10:222
3846401993222,6cyclictest0-21swapper/223:05:012
3846401993222,6cyclictest0-21swapper/223:05:012
3846401993222,6cyclictest0-21swapper/220:25:252
3846396993225,4cyclictest0-21swapper/121:15:151
384639699322,4cyclictest131rcu_preempt20:30:121
3846396993214,5cyclictest3950748-21kworker/u8:1+events_unbound00:18:261
3846392993225,5cyclictest3883003-21kworker/u8:1+events_unbound20:57:530
3846392993223,6cyclictest3938605-21cron23:35:000
3846392993220,4cyclictest131rcu_preempt20:00:250
3846392993219,3cyclictest131rcu_preempt23:35:240
3846392993215,4cyclictest131rcu_preempt22:57:470
3846392993215,4cyclictest131rcu_preempt22:57:470
3846392993215,4cyclictest131rcu_preempt21:43:090
3846392993215,4cyclictest0-21swapper/019:25:510
3846392993215,10cyclictest131rcu_preempt20:35:170
3846392993214,10cyclictest131rcu_preempt22:53:180
3846392993212,4cyclictest0-21swapper/020:22:480
3846392993211,5cyclictest131rcu_preempt19:47:390
3846392993211,5cyclictest131rcu_preempt19:47:390
3846405993123,4cyclictest0-21swapper/300:30:273
3846405993121,6cyclictest0-21swapper/321:25:243
3846405993121,5cyclictest0-21swapper/323:05:143
3846401993123,5cyclictest0-21swapper/200:30:162
3846401993121,6cyclictest0-21swapper/220:00:262
3846401993121,4cyclictest3907538-21kworker/u8:0+events_unbound22:20:162
384640199311,6cyclictest131rcu_preempt23:25:132
3846396993121,6cyclictest0-21swapper/100:40:001
3846392993120,7cyclictest121ktimers/000:05:250
3846392993116,5cyclictest131rcu_preempt23:15:290
3846392993116,4cyclictest131rcu_preempt21:05:190
3846392993115,4cyclictest131rcu_preempt21:51:190
3846392993114,9cyclictest131rcu_preempt21:33:190
3846392993113,9cyclictest131rcu_preempt00:11:140
3846392993113,11cyclictest131rcu_preempt00:05:010
3846405993022,4cyclictest0-21swapper/322:10:163
3846405993022,4cyclictest0-21swapper/300:20:223
3846405993021,6cyclictest3951133-21fschecks_count00:10:143
3846401993023,4cyclictest3943735-21idleruntime-cro23:50:002
3846401993023,4cyclictest3902575-21apt21:50:002
3846401993023,4cyclictest3888500-21auditctl21:10:012
3846401993022,5cyclictest0-21swapper/200:25:002
3846401993022,4cyclictest0-21swapper/222:35:242
3846401993022,4cyclictest0-21swapper/200:00:242
3846401993019,7cyclictest0-21swapper/221:30:262
3846396993022,5cyclictest0-21swapper/119:15:041
3846396993020,7cyclictest0-21swapper/120:45:001
3846396993017,5cyclictest3914266-21kworker/u8:1+events_unbound23:12:041
3846396993016,4cyclictest3883003-21kworker/u8:1+events_unbound21:01:371
3846392993015,5cyclictest131rcu_preempt19:55:170
3846392993012,10cyclictest131rcu_preempt00:26:000
3846392993011,11cyclictest131rcu_preempt19:40:210
3846405992922,3cyclictest0-21swapper/321:35:003
3846405992921,4cyclictest0-21swapper/319:55:263
3846401992922,3cyclictest0-21swapper/222:25:262
3846401992921,3cyclictest0-21swapper/220:45:262
3846401992919,7cyclictest3923055-21munin-plugin-st22:50:002
384639699292,4cyclictest131rcu_preempt00:25:121
3846396992922,4cyclictest3905986-21munin-plugin-st22:00:001
3846396992921,4cyclictest0-21swapper/123:05:241
3846396992915,8cyclictest24-21ksoftirqd/122:35:251
3846405992822,4cyclictest3947138-21cut23:59:593
3846405992822,4cyclictest3947138-21cut23:59:593
384640599282,23cyclictest3854169-21fschecks_time19:30:153
3846405992819,6cyclictest0-21swapper/320:35:013
384640199282,3cyclictest131rcu_preempt20:20:122
3846401992817,7cyclictest0-21swapper/219:55:252
3846396992822,4cyclictest3877716-21kworker/u8:2+events_unbound20:36:371
3846396992820,5cyclictest3945141-21kworker/u8:2+events_unbound00:00:441
3846396992820,5cyclictest0-21swapper/121:45:241
3846396992819,5cyclictest0-21swapper/122:55:011
3846396992819,5cyclictest0-21swapper/122:25:271
3846396992818,6cyclictest3907225-21unixbench_multi22:00:261
3846396992818,6cyclictest3907225-21unixbench_multi22:00:261
3846396992817,7cyclictest0-21swapper/119:40:251
3846396992816,7cyclictest0-21swapper/119:10:191
3846396992814,3cyclictest24-21ksoftirqd/120:00:191
384639299289,4cyclictest131rcu_preempt23:48:580
3846405992719,4cyclictest0-21swapper/320:40:263
3846401992719,5cyclictest0-21swapper/221:25:252
3846401992718,6cyclictest3938194-21grep23:30:262
3846396992714,7cyclictest24-21ksoftirqd/123:15:221
3846405992620,4cyclictest3846386-21cyclictest20:20:013
3846405992620,3cyclictest0-21swapper/321:15:243
3846405992619,4cyclictest0-21swapper/321:40:243
3846405992614,8cyclictest1-21systemd21:36:463
384640199268,4cyclictest131rcu_preempt23:40:152
384640199263,4cyclictest131rcu_preempt19:40:122
3846401992620,3cyclictest0-21swapper/220:50:242
3846401992619,4cyclictest0-21swapper/220:40:272
3846401992617,6cyclictest0-21swapper/222:50:252
3846401992616,6cyclictest0-21swapper/219:15:042
3846396992619,5cyclictest3831847-21kworker/u8:0+events_unbound19:25:201
3846396992618,6cyclictest3884844-21kworker/u8:2+events_unbound21:20:111
3846396992618,5cyclictest0-21swapper/120:10:241
3846396992615,5cyclictest3850567-21kworker/u8:3+events_unbound20:08:341
3846396992615,3cyclictest24-21ksoftirqd/119:55:171
3846396992611,6cyclictest24-21ksoftirqd/123:41:461
3846405992518,4cyclictest0-21swapper/320:35:173
3846405992518,3cyclictest0-21swapper/323:35:143
3846405992517,5cyclictest0-21swapper/319:45:263
3846405992517,5cyclictest0-21swapper/319:45:253
3846405992517,4cyclictest3918343-21fschecks_time22:35:163
3846405992517,4cyclictest0-21swapper/322:25:163
3846405992517,4cyclictest0-21swapper/322:00:163
3846405992517,4cyclictest0-21swapper/322:00:163
3846405992516,6cyclictest3852436-21tune2fs19:25:153
3846405992514,6cyclictest0-21swapper/300:20:003
3846401992515,7cyclictest3886208-21grep21:00:252
3846401992515,6cyclictest0-21swapper/219:30:012
384639699256,12cyclictest24-21ksoftirqd/123:50:171
384639699256,12cyclictest24-21ksoftirqd/123:50:171
3846396992519,3cyclictest0-21swapper/121:45:001
3846396992518,4cyclictest0-21swapper/123:30:251
3846396992512,5cyclictest3933089-21kworker/u8:3+events_unbound23:28:121
384639699251,12cyclictest0-21swapper/121:05:461
3846405992419,3cyclictest145-21jbd2/mmcblk1p2-820:00:173
3846405992416,4cyclictest438-21cron19:25:013
3846405992415,6cyclictest3925470-21sort22:55:203
3846405992415,6cyclictest3925470-21sort22:55:203
3846405992415,6cyclictest0-21swapper/320:25:203
3846405992414,6cyclictest0-21swapper/321:00:173
3846405992411,10cyclictest3885117-21basename21:00:003
3846401992418,4cyclictest3841116-21kworker/u8:1+events_unbound19:13:082
3846401992417,5cyclictest0-21swapper/222:55:152
3846401992417,5cyclictest0-21swapper/222:55:152
384640199241,16cyclictest0-21swapper/223:50:122
384640199241,16cyclictest0-21swapper/223:50:112
384640199241,12cyclictest0-21swapper/221:39:002
384639699246,7cyclictest131rcu_preempt23:55:151
384639699246,7cyclictest131rcu_preempt23:55:141
384639699244,4cyclictest131rcu_preempt19:30:251
3846396992418,4cyclictest3945141-21kworker/u8:2+events_unbound00:05:021
384639699241,6cyclictest3955407-21unixbench_multi00:20:241
3846396992416,4cyclictest0-21swapper/100:30:261
3846396992415,6cyclictest0-21swapper/122:15:241
384639699241,3cyclictest131rcu_preempt22:45:121
3846396992413,3cyclictest24-21ksoftirqd/121:25:141
3846396992411,11cyclictest24-21ksoftirqd/121:50:161
3846405992318,3cyclictest3901634-21ntpq21:45:203
3846405992317,3cyclictest0-21swapper/322:05:213
3846405992317,3cyclictest0-21swapper/322:05:203
3846405992316,4cyclictest0-21swapper/322:30:203
3846405992313,7cyclictest3890272-21sed21:15:013
384640599231,15cyclictest3923436-21cut22:50:163
384640599230,20cyclictest391ktimers/323:40:113
3846401992317,3cyclictest0-21swapper/222:10:222
3846401992316,5cyclictest0-21swapper/223:05:002
3846401992315,5cyclictest311ktimers/222:00:222
3846401992315,5cyclictest311ktimers/222:00:212
3846401992315,5cyclictest0-21swapper/219:35:162
384639699236,8cyclictest131rcu_preempt19:45:241
384639699236,8cyclictest131rcu_preempt19:45:241
3846396992316,5cyclictest3916450-21kworker/u8:3+events_unbound22:30:111
3846396992316,5cyclictest199-21systemd-network23:20:041
3846405992217,2cyclictest0-21swapper/319:35:173
3846405992216,3cyclictest0-21swapper/300:35:173
3846405992215,4cyclictest0-21swapper/323:45:193
3846405992215,4cyclictest0-21swapper/300:00:243
3846405992214,4cyclictest391ktimers/320:05:253
3846401992212,7cyclictest3951542-21ntpq00:10:202
384639699226,6cyclictest131rcu_preempt22:20:191
3846396992216,4cyclictest3934851-21kworker/u8:0+events_unbound23:50:001
3846396992212,6cyclictest0-21swapper/120:20:171
3846405992115,3cyclictest0-21swapper/322:20:243
3846405992114,5cyclictest391ktimers/319:50:263
3846405992114,5cyclictest391ktimers/319:50:263
3846405992110,8cyclictest3905030-21ntp_kernel_err21:55:193
3846401992115,4cyclictest3850567-21kworker/u8:3+events_unbound19:30:202
3846401992112,7cyclictest145-21jbd2/mmcblk1p2-821:40:102
384640599209,8cyclictest3949382-21awk00:05:143
3846405992014,3cyclictest0-21swapper/323:00:203
3846405992014,3cyclictest0-21swapper/323:00:193
3846401992013,3cyclictest0-21swapper/219:45:212
3846401992013,3cyclictest0-21swapper/219:45:202
3846405991912,5cyclictest391ktimers/322:45:203
384640199190,10cyclictest3892373-21apt-get21:20:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional