You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 06:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Mar 15, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12076992826,1cyclictest3655-21latency_hist20:00:013
12063992624,1cyclictest0-21swapper/100:15:561
1207699250,24cyclictest15951-21ntp_states22:52:073
12072992522,2cyclictest25222-21runrttasks23:02:032
1207299250,24cyclictest0-21swapper/223:45:542
12063992523,1cyclictest10259-21apt-get00:20:011
1206399250,24cyclictest0-21swapper/123:54:261
1206399250,24cyclictest0-21swapper/122:44:591
1206399250,24cyclictest0-21swapper/100:03:051
12060992523,1cyclictest22245-21ssh22:57:490
1206099250,24cyclictest0-21swapper/023:38:310
1206099250,24cyclictest0-21swapper/022:50:310
1206099250,24cyclictest0-21swapper/022:10:510
1206099250,24cyclictest0-21swapper/022:04:420
1206099250,23cyclictest0-21swapper/021:15:530
1207699240,24cyclictest5742-21diskmemload22:10:323
1207699240,23cyclictest5742-21diskmemload22:26:193
1207699240,23cyclictest5742-21diskmemload21:51:243
1207699240,23cyclictest15897-21ssh23:53:443
12072992424,0cyclictest0-21swapper/222:52:442
12072992418,5cyclictest4036-21ssh22:10:062
1207299240,3cyclictest3112-21ssh21:37:122
1207299240,23cyclictest5742-21diskmemload21:10:562
1207299240,23cyclictest0-21swapper/221:21:422
1207299240,0cyclictest0-21swapper/222:37:142
12063992423,1cyclictest5742-21diskmemload21:54:411
12063992423,1cyclictest0-21swapper/122:36:471
12063992421,2cyclictest18747-21sh22:55:011
1206399240,24cyclictest0-21swapper/122:25:231
1206399240,24cyclictest0-21swapper/121:47:501
1206399240,23cyclictest12517-21cron23:20:011
1206399240,23cyclictest0-21swapper/121:12:061
12060992422,1cyclictest6001-21sh00:16:010
12060992421,2cyclictest24418-21sh21:57:550
12060992421,2cyclictest0-21swapper/023:17:430
1206099240,23cyclictest5742-21diskmemload00:36:020
1206099240,23cyclictest20951-21ssh00:29:510
1206099240,23cyclictest0-21swapper/000:22:590
1206099240,23cyclictest0-21swapper/000:01:250
1206099240,1cyclictest27842-21diskmemload21:31:010
12076992323,0cyclictest0-21swapper/300:36:553
12076992323,0cyclictest0-21swapper/300:07:423
12076992322,1cyclictest0-21swapper/300:33:223
1207699232,1cyclictest5742-21diskmemload00:17:263
12076992320,2cyclictest30924-21latency_hist20:55:013
12076992320,2cyclictest28145-21perf20:50:013
12076992320,2cyclictest27245-21ssh22:01:553
12076992320,2cyclictest22592-21ssh00:00:303
1207699230,2cyclictest32164-21cron21:35:013
1207699230,2cyclictest28052-21ssh23:03:453
1207699230,23cyclictest5742-21diskmemload23:46:003
1207699230,23cyclictest0-21swapper/321:15:383
1207699230,22cyclictest9514-21ssh22:46:503
1207699230,22cyclictest682-21ssh00:10:453
1207699230,22cyclictest5742-21diskmemload22:35:023
1207699230,22cyclictest5742-21diskmemload21:36:163
1207699230,22cyclictest30163-21ssh23:06:433
1207699230,22cyclictest20712-21sh00:29:293
1207699230,22cyclictest11715-21ssh00:21:143
1207699230,22cyclictest10142-21ssh22:15:583
1207699230,22cyclictest10142-21ssh22:15:583
1207699230,22cyclictest0-21swapper/323:10:073
1207699230,22cyclictest0-21swapper/322:06:113
1207699230,1cyclictest0-21swapper/323:21:343
1207699230,0cyclictest5742-21diskmemload23:16:093
1207699230,0cyclictest5742-21diskmemload21:45:203
12072992323,0cyclictest5742-21diskmemload00:03:502
12072992323,0cyclictest0-21swapper/222:21:132
12072992323,0cyclictest0-21swapper/200:12:392
12072992322,0cyclictest0-21swapper/223:27:022
12072992321,1cyclictest8297-21ssh21:42:122
12072992321,1cyclictest0-21swapper/222:07:572
12072992321,1cyclictest0-21swapper/220:07:022
12072992320,2cyclictest10181-21sh22:16:012
12072992320,2cyclictest10181-21sh22:16:012
12072992319,1cyclictest0-21swapper/221:34:492
12072992318,3cyclictest21907-21diskmemload00:30:522
1207299230,23cyclictest0-21swapper/223:14:022
1207299230,23cyclictest0-21swapper/223:07:082
1207299230,23cyclictest0-21swapper/222:01:562
1207299230,22cyclictest29946-21ssh22:34:432
1207299230,22cyclictest16311-21ssh21:51:112
1207299230,22cyclictest14220-21cut23:52:072
1207299230,22cyclictest11744-21ssh00:21:162
1207299230,21cyclictest151rcu_preempt22:41:042
1207299230,1cyclictest17842-21ssh23:56:262
1207299230,0cyclictest0-21swapper/223:35:452
1207299230,0cyclictest0-21swapper/223:34:372
12063992323,0cyclictest5742-21diskmemload23:20:271
12063992323,0cyclictest0-21swapper/100:36:291
12063992323,0cyclictest0-21swapper/100:07:181
12063992321,1cyclictest0-21swapper/123:37:491
12063992321,1cyclictest0-21swapper/123:00:431
12063992321,1cyclictest0-21swapper/121:39:491
12063992321,1cyclictest0-21swapper/100:10:021
12063992320,2cyclictest10899-21ssh22:17:021
12063992320,2cyclictest10899-21ssh22:17:021
1206399230,23cyclictest0-21swapper/122:06:541
1206399230,23cyclictest0-21swapper/121:27:141
1206399230,22cyclictest6480-21apt-get20:07:021
1206399230,22cyclictest5742-21diskmemload23:44:071
1206399230,22cyclictest5742-21diskmemload00:26:281
1206399230,22cyclictest16550-21sh22:22:021
1206399230,22cyclictest0-21swapper/122:31:041
1206399230,0cyclictest0-21swapper/123:47:361
1206399230,0cyclictest0-21swapper/123:26:551
1206399230,0cyclictest0-21swapper/122:52:251
1206399230,0cyclictest0-21swapper/122:45:071
1206399230,0cyclictest0-21swapper/122:12:131
1206399230,0cyclictest0-21swapper/122:02:131
1206399230,0cyclictest0-21swapper/100:33:301
12060992323,0cyclictest5742-21diskmemload22:44:460
12060992323,0cyclictest5742-21diskmemload22:30:020
12060992323,0cyclictest0-21swapper/021:37:380
12060992323,0cyclictest0-21swapper/021:29:330
12060992323,0cyclictest0-21swapper/000:05:430
12060992321,1cyclictest0-21swapper/022:36:010
12060992321,1cyclictest0-21swapper/021:45:290
1206099230,23cyclictest0-21swapper/023:30:270
1206099230,23cyclictest0-21swapper/022:24:310
1206099230,22cyclictest7393-21diskmemload23:46:120
1206099230,22cyclictest6844-21sh21:11:250
1206099230,22cyclictest1772-21ssh23:40:300
1206099230,22cyclictest0-21swapper/020:42:010
1206099230,1cyclictest17332-21df_inode21:52:030
1206099230,1cyclictest0-21swapper/000:34:460
1206099230,0cyclictest5742-21diskmemload21:21:440
1206099230,0cyclictest0-21swapper/023:28:560
1206099230,0cyclictest0-21swapper/023:06:420
12076992222,0cyclictest0-21swapper/322:58:083
12076992222,0cyclictest0-21swapper/322:20:023
12076992222,0cyclictest0-21swapper/321:56:383
12076992222,0cyclictest0-21swapper/321:43:193
12076992222,0cyclictest0-21swapper/321:27:333
12076992222,0cyclictest0-21swapper/321:10:503
12076992220,1cyclictest606-21dbus-daemon19:35:003
1207699221,1cyclictest0-21swapper/321:22:033
1207699220,21cyclictest5742-21diskmemload22:30:353
1207699220,21cyclictest1726-21ssh23:40:263
1207699220,21cyclictest0-21swapper/323:37:033
1207699220,1cyclictest313-21systemd-journal19:25:003
1207699220,1cyclictest18381-21ssh23:25:063
1207699220,1cyclictest0-21swapper/323:30:213
12072992222,0cyclictest0-21swapper/223:15:282
12072992222,0cyclictest0-21swapper/200:26:192
12072992221,1cyclictest15015-21ssh21:49:282
12072992220,2cyclictest16558-21ssh23:23:212
1207299220,22cyclictest5742-21diskmemload00:35:422
1207299220,21cyclictest5708-21ssh00:15:352
1207299220,21cyclictest28032-21ssh00:06:002
1207299220,21cyclictest20950-21memory22:57:062
1207299220,21cyclictest0-21swapper/222:25:182
1207299220,21cyclictest0-21swapper/221:58:222
1207299220,1cyclictest23570-21ls21:27:022
1207299220,1cyclictest1643-21ssh23:40:192
1207299220,1cyclictest11874-21ssh21:16:052
1207299220,1cyclictest10065-21df_inode22:47:002
1207299220,0cyclictest1662-21apt-get19:57:002
12063992222,0cyclictest0-21swapper/123:11:301
12063992222,0cyclictest0-21swapper/120:42:011
12063992221,0cyclictest0-21swapper/120:50:021
12063992220,1cyclictest5742-21diskmemload23:07:081
12063992220,1cyclictest0-21swapper/121:55:161
12063992220,1cyclictest0-21swapper/121:17:031
1206399220,21cyclictest0-21swapper/123:55:341
1206399220,21cyclictest0-21swapper/121:30:131
1206399220,1cyclictest0-21swapper/121:42:021
12060992222,0cyclictest0-21swapper/023:00:200
12060992220,1cyclictest0-21swapper/022:15:160
12060992220,1cyclictest0-21swapper/022:15:160
12060992220,1cyclictest0-21swapper/022:07:080
12060992219,2cyclictest20056-21sh22:25:060
1206099220,22cyclictest62550irq/127-eno123:53:130
1206099220,22cyclictest0-21swapper/023:10:340
1206099220,22cyclictest0-21swapper/022:46:590
1206099220,21cyclictest21662-21idleruntime-cro19:30:000
1206099220,21cyclictest0-21swapper/023:20:420
1206099220,21cyclictest0-21swapper/020:02:010
1206099220,1cyclictest3777-21apt-get21:07:020
1206099220,1cyclictest0-21swapper/000:11:080
1206099220,0cyclictest0-21swapper/021:40:060
12076992120,0cyclictest0-21swapper/319:52:003
1207699210,21cyclictest0-21swapper/322:40:293
1207699210,20cyclictest0-21swapper/320:32:013
12072992120,0cyclictest31377-21apt-get20:57:012
12072992119,1cyclictest13326-21perf20:20:002
1207299210,20cyclictest8872-21apt-get20:12:002
1207299210,20cyclictest4084-21apt-get20:02:012
1207299210,20cyclictest24481-21apt-get19:37:022
12063992121,0cyclictest12425-21apt-get19:12:011
12063992121,0cyclictest0-21swapper/119:57:001
12063992120,1cyclictest26896-21apt-get19:42:001
12063992119,1cyclictest16282-21apt-get21:20:021
1206399210,21cyclictest0-21swapper/120:30:001
1206399210,20cyclictest0-21swapper/119:37:021
1206399210,0cyclictest17293-21apt-get19:22:011
1206399210,0cyclictest14147-21apt-get20:22:011
1206399210,0cyclictest0-21swapper/123:30:511
12060992121,0cyclictest0-21swapper/020:27:010
12060992119,1cyclictest3684-21basename20:00:010
12060992119,1cyclictest16888-21munin-run19:20:010
12076992019,1cyclictest21332-21apt-get20:37:003
12076992018,1cyclictest13305-21cron20:20:003
12072992020,0cyclictest0-21swapper/219:52:012
12072992020,0cyclictest0-21swapper/219:42:002
12072992017,2cyclictest23348-21idleruntime-cro20:40:012
1207299200,0cyclictest19686-21apt-get19:27:012
12063992020,0cyclictest18938-21apt-get20:32:011
12063992020,0cyclictest0-21swapper/120:12:091
12063992020,0cyclictest0-21swapper/119:32:011
1206399200,0cyclictest0-21swapper/120:17:011
12060992020,0cyclictest31684-21apt-get19:52:010
12060992020,0cyclictest0-21swapper/020:50:020
12060992020,0cyclictest0-21swapper/019:42:000
12060992020,0cyclictest0-21swapper/019:12:000
1206099200,0cyclictest0-21swapper/020:32:010
12076991919,0cyclictest0-21swapper/320:42:013
12076991919,0cyclictest0-21swapper/320:20:023
12072991919,0cyclictest0-21swapper/219:22:022
1207299190,19cyclictest0-21swapper/219:47:002
12063991919,0cyclictest0-21swapper/119:27:011
12060991919,0cyclictest0-21swapper/020:37:000
12076991818,0cyclictest0-21swapper/319:47:003
12076991817,1cyclictest957-21cat21:00:013
12076991816,1cyclictest16540-21apt-get20:27:013
12076991815,2cyclictest10846-21awk20:15:003
12072991818,0cyclictest0-21swapper/220:52:012
12072991818,0cyclictest0-21swapper/220:32:012
12072991818,0cyclictest0-21swapper/220:27:012
12072991818,0cyclictest0-21swapper/219:32:002
12063991818,0cyclictest0-21swapper/120:57:011
12063991818,0cyclictest0-21swapper/120:02:001
12060991818,0cyclictest0-21swapper/020:12:000
12060991818,0cyclictest0-21swapper/019:47:000
12060991818,0cyclictest0-21swapper/019:22:020
1206099180,17cyclictest0-21swapper/020:07:020
12076991715,1cyclictest14843-21/usr/sbin/munin19:17:023
12072991714,2cyclictest28138-21dump-pmu-power20:50:012
12063991716,1cyclictest29289-21apt-get19:47:001
12076991616,0cyclictest0-21swapper/319:12:003
12076991614,1cyclictest6079-21perf20:05:013
12072991614,1cyclictest0-21swapper/220:25:002
1207299160,15cyclictest5482-21sed21:07:112
12063991616,0cyclictest0-21swapper/119:52:011
12060991616,0cyclictest0-21swapper/020:20:020
12072991515,0cyclictest0-21swapper/219:17:022
12063991513,1cyclictest23342-21sendmail-msp20:40:011
1206399150,1cyclictest1824-21sendmail-mta21:01:071
12060991512,2cyclictest28121-21sh20:50:010
1206099150,9cyclictest0-21swapper/019:37:010
1206099150,1cyclictest0-21swapper/021:02:070
12076991414,0cyclictest0-21swapper/319:40:443
1207699140,14cyclictest0-21swapper/319:35:583
1207699140,14cyclictest0-21swapper/319:25:163
1207699140,0cyclictest0-21swapper/321:06:123
1207699140,0cyclictest0-21swapper/321:01:013
12072991414,0cyclictest0-21swapper/219:12:002
1207299140,14cyclictest0-21swapper/220:40:262
1207299140,0cyclictest0-21swapper/221:00:472
1206399140,14cyclictest0-21swapper/121:05:451
1206399140,0cyclictest0-21swapper/119:15:111
12060991414,0cyclictest0-21swapper/020:15:340
12060991414,0cyclictest0-21swapper/019:30:010
1206099140,14cyclictest0-21swapper/020:55:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional