You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 12:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Fri Jul 17, 2026 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3518981994942,5cyclictest3550218-21kworker/u8:1+events_unbound20:42:283
3518978994942,4cyclictest3520617-21munin-plugin-st19:15:012
3518981994640,3cyclictest35662322sleep321:25:113
3518981994635,7cyclictest3548252-21grep20:30:273
3518981994635,7cyclictest3548252-21grep20:30:263
3518981994539,4cyclictest3586374-21kworker/u8:3+events_unbound22:32:463
3518981994539,4cyclictest3586374-21kworker/u8:3+events_unbound22:32:463
3518981994536,4cyclictest3604311-21fschecks_count23:15:163
3518981994527,5cyclictest131rcu_preempt20:59:173
3518968994435,5cyclictest3532037-21grep19:45:170
3518981994331,4cyclictest3610537-21kworker/u8:1+events_unbound00:12:563
3518978994334,6cyclictest3567281-21unixbench_singl21:25:262
3518968994332,7cyclictest3612689-21idleruntime-cro23:40:010
3518981994226,5cyclictest131rcu_preempt22:19:573
3518973994231,7cyclictest3551734-21grep20:40:251
3518973994230,9cyclictest231ktimers/119:30:121
3518981994135,4cyclictest3600301-21kworker/u8:1+events_unbound23:02:503
3518981994133,6cyclictest3561098-21kworker/u8:2+events_unbound21:12:343
3518973994130,7cyclictest3581150-21grep22:05:261
351896899419,24cyclictest131rcu_preempt23:34:270
351896899419,24cyclictest131rcu_preempt23:34:270
3518981994034,4cyclictest3545036-21kworker/u8:1+events_unbound20:22:243
3518981994034,4cyclictest3545036-21kworker/u8:1+events_unbound20:22:233
3518981994029,4cyclictest131rcu_preempt19:26:003
3518981994027,7cyclictest0-21swapper/321:55:003
3518973994030,6cyclictest3594104-21fschecks_count22:45:181
3518981993931,5cyclictest3633369-21sendmail00:39:593
3518981993930,5cyclictest3595832-21fschecks_time22:50:173
3518981993921,4cyclictest131rcu_preempt20:02:473
3518968993928,6cyclictest3606949-21grep23:20:250
3518968993926,10cyclictest121ktimers/022:55:120
3518981993832,4cyclictest3551954-21kworker/u8:2+events_unbound20:52:293
3518981993817,4cyclictest131rcu_preempt19:12:113
3518973993825,9cyclictest3598840-21sendmail23:00:011
3518968993822,10cyclictest131rcu_preempt19:20:220
3518981993731,4cyclictest3546783-21kworker/u8:2+events_unbound20:37:263
3518981993720,4cyclictest131rcu_preempt20:09:023
3518978993729,5cyclictest3612710-21idleruntime-cro23:40:012
3518973993728,5cyclictest0-21swapper/122:00:171
3518981993630,4cyclictest3569244-21kworker/u8:1+events_unbound21:47:413
351898199362,22cyclictest0-21swapper/319:45:183
3518981993621,5cyclictest131rcu_preempt23:25:173
3518981993621,5cyclictest131rcu_preempt23:25:163
3518981993618,11cyclictest3572760-21kworker/u8:2-flush-179:9621:44:273
3518973993625,7cyclictest3613323-21cut23:40:161
3518973993625,7cyclictest0-21swapper/119:15:011
3518981993527,6cyclictest3534758-21kworker/u8:0+events_unbound20:12:223
3518981993525,6cyclictest0-21swapper/323:40:013
3518981993524,6cyclictest0-21swapper/321:35:263
3518981993523,8cyclictest438-21cron22:30:003
3518981993523,8cyclictest438-21cron22:30:003
3518981993520,4cyclictest131rcu_preempt19:55:003
3518981993517,11cyclictest3595086-21kworker/u8:0-events_freezable_power_22:48:023
3518981993516,6cyclictest3593360-21kworker/u8:1+events_unbound22:43:253
3518981993515,11cyclictest3558371-21kworker/u8:0-events_freezable_power_21:00:273
3518978993528,4cyclictest0-21swapper/220:10:162
3518978993526,6cyclictest3626543-21cron00:19:592
3518973993528,5cyclictest3512021-21kworker/u8:0+events_unbound19:17:101
3518968993517,5cyclictest131rcu_preempt23:55:020
3518981993428,4cyclictest3534758-21kworker/u8:0+events_unbound20:27:243
3518981993428,4cyclictest3534758-21kworker/u8:0+events_unbound20:27:243
3518981993426,6cyclictest3565649-21kworker/u8:2+events_unbound21:22:353
3518981993419,5cyclictest131rcu_preempt19:55:543
3518981993418,5cyclictest131rcu_preempt23:05:163
3518981993418,3cyclictest0-21swapper/322:14:513
3518981993415,5cyclictest131rcu_preempt20:50:013
3518981993414,12cyclictest131rcu_preempt23:40:263
3518981993411,5cyclictest0-21swapper/300:10:013
3518981993410,12cyclictest40-21ksoftirqd/323:35:013
3518981993410,12cyclictest40-21ksoftirqd/323:35:013
3518978993422,6cyclictest0-21swapper/221:55:012
3518973993424,7cyclictest0-21swapper/121:25:121
3518973993424,6cyclictest3534591-21tail19:50:251
351896899345,7cyclictest11-21ksoftirqd/021:35:010
351896899343,14cyclictest11-21ksoftirqd/019:25:180
351896899341,21cyclictest11-21ksoftirqd/020:05:010
3518981993326,5cyclictest3588317-21kworker/u8:0+events_unbound22:36:403
3518981993323,6cyclictest0-21swapper/319:40:253
3518981993323,4cyclictest131rcu_preempt22:23:233
3518981993323,4cyclictest131rcu_preempt22:23:233
3518981993316,5cyclictest131rcu_preempt19:15:263
3518981993315,4cyclictest131rcu_preempt21:55:143
3518981993314,11cyclictest131rcu_preempt22:09:533
3518978993326,4cyclictest3552139-21cron20:44:592
3518978993326,3cyclictest0-21swapper/223:25:002
3518973993326,5cyclictest3569244-21kworker/u8:1+events_unbound21:42:381
3518968993329,2cyclictest0-21swapper/020:29:120
3518968993329,2cyclictest0-21swapper/020:29:120
351898199329,12cyclictest40-21ksoftirqd/300:00:013
3518981993225,3cyclictest0-21swapper/323:55:013
3518981993221,5cyclictest131rcu_preempt23:20:023
3518981993219,5cyclictest131rcu_preempt19:20:153
3518981993214,4cyclictest131rcu_preempt23:47:203
3518978993226,3cyclictest0-21swapper/222:45:172
3518978993225,4cyclictest0-21swapper/221:55:252
3518978993225,4cyclictest0-21swapper/200:35:252
3518978993222,6cyclictest3608637-21unixbench_singl23:25:272
3518978993222,6cyclictest3608637-21unixbench_singl23:25:272
3518978993221,7cyclictest0-21swapper/223:50:252
351896899326,7cyclictest11-21ksoftirqd/021:10:010
351896899322,5cyclictest11-21ksoftirqd/000:15:000
3518968993225,3cyclictest0-21swapper/023:49:590
3518968993224,4cyclictest0-21swapper/019:55:250
3518968993220,5cyclictest0-21swapper/022:28:040
3518968993220,5cyclictest0-21swapper/022:28:040
3518968993217,8cyclictest131rcu_preempt22:10:140
3518968993216,3cyclictest131rcu_preempt00:38:390
351898199319,10cyclictest40-21ksoftirqd/319:30:263
351898199318,11cyclictest40-21ksoftirqd/321:05:263
351898199315,14cyclictest40-21ksoftirqd/300:18:263
3518981993123,4cyclictest0-21swapper/320:15:283
3518981993121,7cyclictest3598813-21sh23:00:003
3518981993112,5cyclictest131rcu_preempt00:25:213
3518978993121,6cyclictest3593264-21grep22:40:262
3518973993122,5cyclictest3546607-21unixbench_singl20:25:271
3518973993122,5cyclictest3546607-21unixbench_singl20:25:261
3518973993122,5cyclictest0-21swapper/119:50:011
351896899314,5cyclictest11-21ksoftirqd/021:05:000
3518968993123,5cyclictest0-21swapper/023:50:260
3518968993122,6cyclictest3534610-21unixbench_multi19:50:250
3518968993121,7cyclictest3620971-21grep00:00:290
3518968993121,7cyclictest121ktimers/022:55:010
3518968993121,6cyclictest3527778-21grep19:30:260
3518968993121,4cyclictest131rcu_preempt19:19:580
3518968993120,7cyclictest0-21swapper/023:30:000
3518968993120,7cyclictest0-21swapper/023:29:590
3518968993117,3cyclictest131rcu_preempt20:30:210
3518968993117,3cyclictest131rcu_preempt20:30:200
3518968993115,5cyclictest131rcu_preempt21:49:270
3518968993112,11cyclictest145-21jbd2/mmcblk1p2-822:33:060
3518968993112,11cyclictest145-21jbd2/mmcblk1p2-822:33:060
351898199308,11cyclictest40-21ksoftirqd/300:35:003
3518981993016,4cyclictest131rcu_preempt21:16:453
3518981993014,4cyclictest131rcu_preempt23:10:153
3518981993013,4cyclictest131rcu_preempt21:34:593
3518981993012,8cyclictest131rcu_preempt00:20:163
3518981993010,3cyclictest0-21swapper/322:00:243
3518973993023,4cyclictest0-21swapper/122:50:271
3518973993022,5cyclictest3536776-21grep20:00:011
3518973993022,4cyclictest0-21swapper/120:50:261
351896899305,8cyclictest11-21ksoftirqd/000:30:240
3518968993024,4cyclictest3572371-21unixbench_multi21:40:260
3518968993024,4cyclictest0-21swapper/023:15:120
3518968993022,5cyclictest0-21swapper/020:25:000
3518968993022,5cyclictest0-21swapper/020:24:590
3518968993020,7cyclictest3567705-21idleruntime-cro21:30:010
3518968993020,7cyclictest121ktimers/019:10:180
3518968993015,8cyclictest131rcu_preempt20:44:330
3518968993014,5cyclictest131rcu_preempt22:00:130
3518968993013,9cyclictest11-21ksoftirqd/022:21:010
3518968993013,9cyclictest11-21ksoftirqd/022:21:000
3518968993011,9cyclictest131rcu_preempt21:15:240
3518968993011,5cyclictest3518958-21cyclictest22:05:170
3518968993011,10cyclictest131rcu_preempt20:35:150
3518981992911,4cyclictest131rcu_preempt19:35:193
3518978992921,5cyclictest3590091-21munin-plugin-st22:35:002
3518978992921,5cyclictest3590091-21munin-plugin-st22:35:002
3518978992921,4cyclictest0-21swapper/223:40:162
3518978992921,4cyclictest0-21swapper/222:05:272
3518978992921,4cyclictest0-21swapper/200:30:002
3518978992920,6cyclictest53-21kcompactd019:50:212
3518973992923,4cyclictest3574514-21kworker/u8:0+events_unbound22:17:431
3518973992922,5cyclictest3534758-21kworker/u8:0+events_unbound20:15:201
3518968992922,3cyclictest0-21swapper/023:44:590
3518968992921,5cyclictest121ktimers/022:38:570
3518968992919,7cyclictest121ktimers/021:36:290
3518968992919,7cyclictest121ktimers/000:05:260
3518968992914,4cyclictest131rcu_preempt20:10:120
3518968992914,3cyclictest131rcu_preempt20:05:220
3518968992913,4cyclictest131rcu_preempt20:45:270
3518968992913,4cyclictest131rcu_preempt20:15:210
3518968992912,8cyclictest0-21swapper/020:56:160
3518968992911,4cyclictest131rcu_preempt23:03:260
3518968992910,12cyclictest151rcuc/019:35:240
3518978992821,4cyclictest3536773-21cron20:00:002
3518978992820,4cyclictest0-21swapper/223:10:162
3518978992819,6cyclictest0-21swapper/221:15:252
3518973992820,4cyclictest0-21swapper/123:10:271
3518973992820,4cyclictest0-21swapper/122:35:001
3518973992820,4cyclictest0-21swapper/122:34:591
3518973992820,4cyclictest0-21swapper/121:00:261
351896899289,6cyclictest3518958-21cyclictest00:25:310
351896899289,10cyclictest131rcu_preempt21:50:220
351896899288,4cyclictest0-21swapper/022:45:180
351896899287,8cyclictest3601542-21switchtime23:05:250
351896899284,5cyclictest11-21ksoftirqd/000:15:250
351896899283,5cyclictest11-21ksoftirqd/019:40:230
351896899282,7cyclictest11-21ksoftirqd/022:40:140
3518968992822,3cyclictest0-21swapper/020:50:260
3518968992818,7cyclictest121ktimers/021:10:110
3518968992810,4cyclictest0-21swapper/022:19:350
3518978992721,4cyclictest3610537-21kworker/u8:1+events_unbound00:02:552
3518978992720,4cyclictest3559166-21cron21:05:002
3518973992721,4cyclictest3551954-21kworker/u8:2+events_unbound20:47:271
3518973992719,5cyclictest3628285-21wc00:25:011
351896899279,4cyclictest131rcu_preempt23:10:160
3518968992712,4cyclictest131rcu_preempt21:20:130
3518968992710,5cyclictest131rcu_preempt21:55:230
3518978992620,4cyclictest3583644-21grep22:15:172
3518978992620,3cyclictest0-21swapper/223:45:252
3518973992620,3cyclictest0-21swapper/100:00:261
3518973992618,4cyclictest0-21swapper/123:35:221
3518973992618,4cyclictest0-21swapper/120:35:161
3518973992618,4cyclictest0-21swapper/100:10:271
3518968992619,4cyclictest121ktimers/000:20:180
3518978992518,4cyclictest3601308-21ntpq23:05:212
3518978992515,7cyclictest3611027-21cat23:35:012
3518978992515,7cyclictest3611027-21cat23:35:002
3518973992519,4cyclictest3588317-21kworker/u8:0+events_unbound22:37:471
3518973992519,3cyclictest0-21swapper/121:20:171
3518973992518,4cyclictest0-21swapper/121:15:271
3518973992518,4cyclictest0-21swapper/120:10:001
3518973992517,5cyclictest0-21swapper/119:45:001
3518973992517,4cyclictest3547386-21fschecks_count20:30:171
3518973992517,4cyclictest3547386-21fschecks_count20:30:161
3518973992515,7cyclictest3583254-21auditctl22:14:591
3518973992513,9cyclictest3586793-21cut22:25:011
3518973992513,9cyclictest3586793-21cut22:25:011
3518978992418,3cyclictest0-21swapper/222:55:262
3518978992417,5cyclictest3588317-21kworker/u8:0+events_unbound22:35:052
3518978992417,4cyclictest3619710-21sh00:00:012
3518978992417,3cyclictest0-21swapper/219:20:142
3518978992416,5cyclictest35426802sleep220:15:252
3518978992415,6cyclictest3574514-21kworker/u8:0+events_unbound22:12:422
3518978992415,6cyclictest0-21swapper/222:00:212
351897899241,12cyclictest0-21swapper/220:02:432
3518973992418,4cyclictest3598569-21kworker/u8:2+events_unbound23:07:511
3518973992417,5cyclictest3569244-21kworker/u8:1+events_unbound21:50:211
3518973992417,4cyclictest231ktimers/123:00:391
3518973992417,4cyclictest0-21swapper/100:35:251
3518973992416,5cyclictest0-21swapper/119:25:151
3518978992317,3cyclictest0-21swapper/200:30:242
3518978992316,4cyclictest3597109-21sh22:55:012
3518978992316,4cyclictest0-21swapper/220:35:262
3518978992316,3cyclictest0-21swapper/223:20:002
3518978992315,5cyclictest0-21swapper/219:15:022
3518978992314,6cyclictest0-21swapper/219:45:012
351897399239,3cyclictest0-21swapper/123:57:181
351897399234,5cyclictest131rcu_preempt00:05:151
3518973992317,3cyclictest3615056-21fschecks_time23:45:161
3518973992316,5cyclictest0-21swapper/121:05:241
3518973992316,3cyclictest0-21swapper/122:27:441
3518973992316,3cyclictest0-21swapper/122:27:441
3518973992315,4cyclictest3609072-21munin-plugin-st23:30:001
3518973992315,4cyclictest3609072-21munin-plugin-st23:30:001
3518973992314,5cyclictest0-21swapper/119:40:011
3518978992217,2cyclictest0-21swapper/220:25:192
3518978992217,2cyclictest0-21swapper/220:25:182
3518978992216,3cyclictest0-21swapper/222:25:212
3518978992216,3cyclictest0-21swapper/222:25:212
3518978992216,3cyclictest0-21swapper/220:55:162
3518978992216,3cyclictest0-21swapper/200:05:242
3518978992215,4cyclictest438-21cron20:55:012
3518978992215,4cyclictest311ktimers/220:25:012
3518978992215,4cyclictest311ktimers/220:25:012
3518978992213,6cyclictest3552470-21ls20:45:142
3518978992212,8cyclictest3626983-21cut00:20:162
3518978992212,7cyclictest311ktimers/221:10:102
3518973992216,3cyclictest0-21swapper/123:50:241
351897399221,4cyclictest3576219-21apt-get21:55:041
3518978992113,6cyclictest311ktimers/221:30:172
3518978992113,5cyclictest3586685-21ls22:25:002
3518978992113,5cyclictest3586685-21ls22:25:002
3518978992113,5cyclictest311ktimers/221:20:172
3518978992112,5cyclictest0-21swapper/200:10:272
3518978992111,7cyclictest3560883-21ls21:09:592
3518978992110,8cyclictest3547360-21df_inode20:30:162
3518978992110,8cyclictest3547360-21df_inode20:30:162
351897399217,3cyclictest131rcu_preempt21:30:151
351897399213,5cyclictest131rcu_preempt21:45:141
3518973992115,4cyclictest3520523-21kworker/u8:2+events_unbound19:22:131
3518973992115,3cyclictest0-21swapper/123:20:111
3518973992113,5cyclictest3627951-21kworker/u8:0+events_unbound00:25:061
3518973992113,5cyclictest0-21swapper/121:35:171
351897899201,5cyclictest3529524-21unixbench_singl19:35:252
3518978992015,3cyclictest3521034-21munin-node19:45:142
3518978992014,3cyclictest0-21swapper/221:45:182
3518978992014,3cyclictest0-21swapper/219:25:242
3518978992013,5cyclictest311ktimers/223:00:142
3518978992013,5cyclictest311ktimers/221:40:272
3518978992013,5cyclictest311ktimers/219:30:102
3518978992012,4cyclictest0-21swapper/220:10:012
3518978992011,6cyclictest3570551-21grep21:35:262
351897399201,4cyclictest0-21swapper/123:30:271
351897399201,4cyclictest0-21swapper/123:30:271
3518973992014,3cyclictest0-21swapper/122:40:171
3518973992014,3cyclictest0-21swapper/100:30:141
351897399199,7cyclictest3625983-21wc00:15:271
351897399198,8cyclictest3544740-21cat20:20:261
351897399198,8cyclictest3544740-21cat20:20:261
3518973991912,5cyclictest231ktimers/120:00:121
3518973991911,5cyclictest3560663-21kworker/u8:0+events_unbound21:13:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional