You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 11:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Aug 02, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
907699270,26cyclictest0-21swapper/323:35:003
9076992621,4cyclictest0-21swapper/321:14:513
907699260,26cyclictest0-21swapper/323:48:043
907699260,22cyclictest10324-21ssh22:14:303
907699260,0cyclictest0-21swapper/322:47:043
907199260,25cyclictest622-21gdbus20:05:011
9076992523,2cyclictest0-21swapper/321:24:563
9076992522,3cyclictest0-21swapper/323:57:163
9076992521,3cyclictest2369-21diskmemload21:26:233
907699250,24cyclictest261-21jbd2/nvme0n1p2-20:55:013
907699250,24cyclictest2369-21diskmemload22:18:533
907699250,24cyclictest0-21swapper/323:54:463
907699250,24cyclictest0-21swapper/321:45:493
907699250,24cyclictest0-21swapper/300:01:393
907699250,21cyclictest2369-21diskmemload23:26:513
9074992522,2cyclictest2369-21diskmemload23:25:522
907499250,24cyclictest11822-21irqstats21:18:462
907499250,1cyclictest0-21swapper/200:38:032
907199250,24cyclictest10498-21ssh22:14:521
907199250,24cyclictest0-21swapper/122:25:231
907199250,24cyclictest0-21swapper/119:15:021
9069992523,1cyclictest0-21swapper/023:23:490
906999250,24cyclictest7706-21/usr/sbin/munin00:08:520
9076992423,0cyclictest10970-21apt-get21:18:423
9076992422,2cyclictest0-21swapper/300:35:483
9076992421,3cyclictest0-21swapper/322:41:543
9076992420,3cyclictest2369-21diskmemload22:28:043
9076992420,3cyclictest0-21swapper/323:15:593
907699240,24cyclictest0-21swapper/300:07:273
907699240,23cyclictest0-21swapper/322:01:413
907699240,21cyclictest2369-21diskmemload22:37:323
907699240,1cyclictest11123-21ssh23:43:413
907699240,0cyclictest2369-21diskmemload22:57:113
907699240,0cyclictest0-21swapper/321:52:563
907699240,0cyclictest0-21swapper/321:44:583
9074992424,0cyclictest2369-21diskmemload22:44:432
9074992419,4cyclictest0-21swapper/222:32:372
9074992419,4cyclictest0-21swapper/222:21:252
907499240,24cyclictest0-21swapper/223:18:582
907499240,24cyclictest0-21swapper/223:10:022
907499240,24cyclictest0-21swapper/221:25:542
907499240,23cyclictest90732sleep200:10:012
907499240,23cyclictest27353-21ssh23:58:072
907499240,23cyclictest22111-21/usr/sbin/munin23:53:452
907499240,1cyclictest17560-21ssh22:50:072
9071992424,0cyclictest0-21swapper/123:17:451
9071992423,1cyclictest4775-21ssh00:06:291
9071992423,1cyclictest0-21swapper/123:06:041
907199242,22cyclictest20406-21kworker/u8:021:57:021
9071992422,1cyclictest23111-21idleruntime21:28:431
907199240,24cyclictest0-21swapper/123:14:581
907199240,24cyclictest0-21swapper/122:51:041
907199240,24cyclictest0-21swapper/121:42:471
907199240,24cyclictest0-21swapper/100:19:241
907199240,24cyclictest0-21swapper/100:12:351
907199240,23cyclictest28369-21basename23:00:001
907199240,23cyclictest2369-21diskmemload23:50:471
907199240,23cyclictest2369-21diskmemload00:32:021
9069992423,1cyclictest2369-21diskmemload21:24:100
9069992423,1cyclictest23690-21/usr/sbin/munin23:53:500
9069992422,1cyclictest62550irq/127-eno121:48:490
9069992422,1cyclictest0-21swapper/023:41:590
906999240,24cyclictest2369-21diskmemload22:50:270
906999240,24cyclictest0-21swapper/000:35:260
906999240,24cyclictest0-21swapper/000:25:450
906999240,23cyclictest0-21swapper/022:43:120
906999240,23cyclictest0-21swapper/022:35:570
906999240,0cyclictest0-21swapper/023:16:170
9076992323,0cyclictest0-21swapper/323:20:563
9076992323,0cyclictest0-21swapper/321:35:473
9076992320,3cyclictest0-21swapper/323:01:513
9076992320,2cyclictest30377-21sh00:29:003
9076992320,2cyclictest2369-21diskmemload00:17:303
9076992320,2cyclictest0-21swapper/322:52:563
9076992319,3cyclictest0-21swapper/300:14:533
907699230,23cyclictest0-21swapper/322:05:123
907699230,23cyclictest0-21swapper/300:32:233
907699230,22cyclictest9611-21ssh23:12:413
907699230,0cyclictest2369-21diskmemload23:36:443
907699230,0cyclictest2369-21diskmemload23:05:463
907699230,0cyclictest0-21swapper/300:21:313
9074992323,0cyclictest2369-21diskmemload22:15:422
9074992323,0cyclictest0-21swapper/223:05:372
9074992323,0cyclictest0-21swapper/221:21:342
9074992323,0cyclictest0-21swapper/221:12:102
9074992323,0cyclictest0-21swapper/220:28:402
9074992321,1cyclictest2369-21diskmemload00:01:022
9074992321,1cyclictest0-21swapper/223:30:412
9074992320,2cyclictest23759-21sh22:55:552
907499230,2cyclictest19701-21ssh23:21:102
907499230,23cyclictest0-21swapper/222:47:342
907499230,23cyclictest0-21swapper/221:43:322
907499230,22cyclictest29904-21ssh23:01:202
907499230,22cyclictest24126-21ssh22:28:132
907499230,22cyclictest2256-21ssh22:36:532
907499230,22cyclictest22156-21sh21:56:562
907499230,22cyclictest10028-21sh23:42:102
907499230,0cyclictest2369-21diskmemload21:30:352
907499230,0cyclictest0-21swapper/223:35:042
907499230,0cyclictest0-21swapper/200:18:052
9071992323,0cyclictest2369-21diskmemload22:18:051
9071992323,0cyclictest2369-21diskmemload21:45:081
9071992323,0cyclictest0-21swapper/123:57:491
9071992323,0cyclictest0-21swapper/100:24:201
9071992323,0cyclictest0-21swapper/100:02:251
9071992322,1cyclictest2369-21diskmemload22:33:591
9071992322,1cyclictest2369-21diskmemload21:13:461
9071992321,2cyclictest16009-21ssh21:51:111
9071992321,1cyclictest1438-21sh22:07:191
9071992321,1cyclictest13805-21munin-run23:45:001
907199230,23cyclictest2369-21diskmemload22:20:341
907199230,22cyclictest7770-21ssh22:41:511
907199230,22cyclictest29901-21ssh23:30:141
907199230,22cyclictest25784-21ssh21:30:371
907199230,22cyclictest2369-21diskmemload23:01:101
907199230,0cyclictest2369-21diskmemload23:25:091
907199230,0cyclictest0-21swapper/123:36:201
9069992323,0cyclictest2369-21diskmemload00:12:500
9069992323,0cyclictest0-21swapper/022:26:370
9069992323,0cyclictest0-21swapper/021:11:510
9069992323,0cyclictest0-21swapper/000:30:210
9069992323,0cyclictest0-21swapper/000:20:560
9069992322,1cyclictest2369-21diskmemload21:31:140
9069992321,1cyclictest0-21swapper/023:05:010
9069992321,1cyclictest0-21swapper/022:24:020
906999232,0cyclictest62550irq/127-eno122:04:120
906999230,23cyclictest0-21swapper/023:30:070
906999230,23cyclictest0-21swapper/019:53:420
906999230,22cyclictest7846-21sh23:10:090
906999230,22cyclictest26124-21sh23:27:360
906999230,22cyclictest2394-21idleruntime-cro21:10:010
906999230,22cyclictest15842-21ssh21:50:490
906999230,22cyclictest14209-21ls19:20:010
906999230,22cyclictest10612-21cron22:15:000
906999230,22cyclictest0-21swapper/023:35:360
906999230,22cyclictest0-21swapper/023:00:460
906999230,1cyclictest14564-21ssh22:48:420
906999230,0cyclictest0-21swapper/021:35:080
9076992222,0cyclictest2369-21diskmemload22:30:053
9076992220,1cyclictest20329-21apt20:40:003
9076992220,1cyclictest16504-21ssh22:20:133
9076992220,1cyclictest0-21swapper/319:13:413
9076992220,1cyclictest0-21swapper/319:13:413
9076992219,3cyclictest0-21swapper/321:32:213
907699220,21cyclictest14253-21munin-run19:20:013
9074992222,0cyclictest0-21swapper/221:51:472
907499220,22cyclictest0-21swapper/200:30:552
907499220,21cyclictest31198-21sh21:35:232
907499220,21cyclictest26757-21ssh22:00:472
907499220,21cyclictest0-21swapper/222:11:052
907499220,21cyclictest0-21swapper/200:20:452
907499220,1cyclictest31087-21perf19:55:002
907499220,1cyclictest14257-21ssh21:48:592
907499220,1cyclictest1131-21perf20:00:002
907499220,1cyclictest0-21swapper/222:06:002
907499220,0cyclictest0-21swapper/221:03:392
9071992220,1cyclictest968-21sh21:38:251
9071992220,1cyclictest26492-21ssh22:00:191
907199220,22cyclictest14824-21ssh23:46:141
907199220,21cyclictest4306-21apt-get00:35:021
907199220,21cyclictest2369-21diskmemload00:25:391
907199220,21cyclictest1293-21ssh22:35:401
907199220,21cyclictest0-21swapper/121:15:071
907199220,1cyclictest9575-21apt-get19:13:411
907199220,1cyclictest9575-21apt-get19:13:411
907199220,1cyclictest0-21swapper/121:21:221
9069992222,0cyclictest0-21swapper/022:15:310
9069992220,1cyclictest2369-21diskmemload23:46:380
9069992220,1cyclictest0-21swapper/022:05:340
906999221,20cyclictest31623-21apt-get19:58:400
906999220,22cyclictest0-21swapper/022:55:570
906999220,22cyclictest0-21swapper/000:03:030
906999220,21cyclictest2369-21diskmemload21:25:280
906999220,21cyclictest2369-21diskmemload00:15:240
906999220,1cyclictest5246-21ssh21:41:360
906999220,1cyclictest28630-21idleruntime-cro19:50:000
906999220,1cyclictest21584-21sh21:56:130
906999220,1cyclictest0-21swapper/020:45:000
9076992119,2cyclictest0-21swapper/320:58:403
9076992119,1cyclictest13143-21perf20:25:003
907699210,21cyclictest0-21swapper/320:48:423
907699210,21cyclictest0-21swapper/319:38:413
907699210,20cyclictest0-21swapper/320:13:443
907699210,1cyclictest13686-21apt-get20:28:403
9074992120,0cyclictest8874-21apt-get20:18:432
907499210,20cyclictest0-21swapper/219:43:422
907499210,1cyclictest20339-21latency_hist20:40:002
907499210,0cyclictest0-21swapper/220:33:422
9071992121,0cyclictest26783-21apt-get19:48:401
9071992121,0cyclictest0-21swapper/119:38:411
9071992121,0cyclictest0-21swapper/119:25:011
907199210,21cyclictest0-21swapper/123:20:181
907199210,20cyclictest19577-21apt-get19:33:391
907199210,20cyclictest0-21swapper/120:43:431
907199210,20cyclictest0-21swapper/119:53:421
906999210,0cyclictest0-21swapper/020:33:420
9076992020,0cyclictest0-21swapper/319:58:413
907699200,0cyclictest0-21swapper/319:53:413
9074992020,0cyclictest23292-21apt-get20:48:422
9074992019,0cyclictest11295-21apt-get20:23:412
907499200,0cyclictest28080-21apt-get20:58:402
907499200,0cyclictest0-21swapper/219:33:392
9069992020,0cyclictest0-21swapper/021:03:390
9076991919,0cyclictest0-21swapper/320:43:433
9076991918,1cyclictest0-21swapper/319:43:423
9076991917,1cyclictest0-21swapper/319:35:003
907699190,2cyclictest0-21swapper/320:03:403
907699190,0cyclictest16086-21apt-get20:33:423
9074991919,0cyclictest14783-21apt-get19:23:412
9071991919,0cyclictest17179-21apt-get19:28:401
9071991918,1cyclictest4078-21apt-get20:08:411
9069991919,0cyclictest0-21swapper/020:23:400
9069991919,0cyclictest0-21swapper/019:28:400
9069991919,0cyclictest0-21swapper/019:13:410
9069991919,0cyclictest0-21swapper/019:13:410
9076991818,0cyclictest0-21swapper/319:28:403
9076991810,7cyclictest32357-21grep21:05:013
9074991817,0cyclictest0-21swapper/220:08:412
9071991818,0cyclictest6479-21apt-get20:13:441
9071991818,0cyclictest0-21swapper/121:08:411
9071991818,0cyclictest0-21swapper/120:53:411
9069991818,0cyclictest0-21swapper/020:38:390
9069991818,0cyclictest0-21swapper/020:28:400
9069991818,0cyclictest0-21swapper/020:08:400
9069991818,0cyclictest0-21swapper/019:38:400
9071991717,0cyclictest24393-21apt-get19:43:421
9071991717,0cyclictest0-21swapper/120:58:401
9071991716,1cyclictest32342-21cron21:05:011
9069991717,0cyclictest0-21swapper/020:58:400
9069991717,0cyclictest0-21swapper/020:48:420
9069991717,0cyclictest0-21swapper/020:13:440
9069991715,1cyclictest0-21swapper/020:55:010
9076991616,0cyclictest0-21swapper/321:08:423
9076991616,0cyclictest0-21swapper/320:18:433
907699160,1cyclictest28782-21munin-run19:50:003
9074991616,0cyclictest1678-21apt-get20:03:402
9074991616,0cyclictest0-21swapper/219:48:402
9074991615,1cyclictest12366-21apt-get19:18:412
907499160,15cyclictest2410-21awk21:10:012
907199160,1cyclictest18903-21diskstats20:38:401
907199160,15cyclictest9053-21grep20:18:441
907199160,15cyclictest14736-21ntp_states20:28:461
9069991616,0cyclictest0-21swapper/020:03:400
9069991616,0cyclictest0-21swapper/019:23:400
906999160,15cyclictest26111-21users19:43:520
907699150,5cyclictest0-21swapper/320:08:483
9074991515,0cyclictest25688-21apt-get20:53:412
9069991515,0cyclictest0-21swapper/020:18:430
907699140,14cyclictest0-21swapper/319:20:333
9074991414,0cyclictest0-21swapper/220:10:492
907499140,14cyclictest0-21swapper/220:40:102
907499140,14cyclictest0-21swapper/219:36:082
907499140,14cyclictest0-21swapper/219:10:312
907499140,14cyclictest0-21swapper/219:10:312
907499140,0cyclictest0-21swapper/219:26:112
9071991414,0cyclictest0-21swapper/119:56:091
907199140,14cyclictest0-21swapper/120:45:261
906999140,0cyclictest0-21swapper/019:30:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional