You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 08:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 16, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
972599273,23cyclictest8993-21cron00:35:012
971999260,25cyclictest26744-21grep00:22:200
972799250,24cyclictest6458-21ssh00:04:163
972799250,24cyclictest19308-21kworker/u8:100:27:123
972799250,24cyclictest0-21swapper/322:31:103
972799250,1cyclictest0-21swapper/321:50:003
972599250,24cyclictest8647-21ssh22:11:332
972199250,2cyclictest5104-21ssh23:05:591
972199250,24cyclictest18446-21cron22:20:011
972199250,24cyclictest0-21swapper/120:12:141
971999250,24cyclictest3752-21diskmemload22:16:070
971999250,24cyclictest3752-21diskmemload22:01:400
971999250,24cyclictest0-21swapper/023:30:470
971999250,1cyclictest0-21swapper/021:28:220
9727992423,0cyclictest0-21swapper/323:47:163
972799242,21cyclictest31619-21sh23:00:373
972799240,24cyclictest0-21swapper/322:50:403
972799240,23cyclictest0-21swapper/321:12:393
972799240,1cyclictest3752-21diskmemload21:53:183
9725992422,2cyclictest29556-21ssh23:56:422
9725992422,1cyclictest10503-21apt-get22:42:162
9725992421,2cyclictest22763-21rm00:18:012
972599240,24cyclictest0-21swapper/222:54:122
972599240,1cyclictest3752-21diskmemload00:08:422
9721992424,0cyclictest0-21swapper/100:36:441
9721992423,0cyclictest0-21swapper/121:12:331
9721992422,1cyclictest4541-21munin-run20:05:011
972199240,24cyclictest3752-21diskmemload00:13:171
972199240,23cyclictest3752-21diskmemload21:25:281
972199240,23cyclictest3752-21diskmemload00:10:001
972199240,23cyclictest28267-21ssh22:28:041
972199240,1cyclictest30024-21ssh22:30:041
972199240,0cyclictest0-21swapper/123:50:561
9719992423,1cyclictest0-21swapper/023:44:400
971999240,23cyclictest3752-21diskmemload00:01:440
971999240,23cyclictest0-21swapper/022:29:520
971999240,1cyclictest170302sleep021:21:510
971999240,1cyclictest0-21swapper/000:14:270
9727992323,0cyclictest0-21swapper/323:33:003
9727992323,0cyclictest0-21swapper/323:26:253
9727992323,0cyclictest0-21swapper/321:38:303
9727992321,1cyclictest31080-21hddtemp_smartct22:02:203
9727992321,1cyclictest0-21swapper/322:36:483
9727992320,2cyclictest28835-21rm23:55:413
9727992320,2cyclictest21110-21idleruntime-cro21:25:013
9727992320,2cyclictest12872-21awk21:17:203
972799230,23cyclictest0-21swapper/300:31:023
972799230,22cyclictest9371-21/usr/sbin/munin21:42:283
972799230,22cyclictest3752-21diskmemload21:56:263
972799230,22cyclictest23906-21rm23:51:213
972799230,22cyclictest23464-21cstates21:27:173
972799230,22cyclictest19276-21ssh22:20:433
972799230,22cyclictest0-21swapper/323:40:003
972799230,1cyclictest0-21swapper/321:33:583
972799230,0cyclictest0-21swapper/323:20:103
972799230,0cyclictest0-21swapper/322:57:223
972799230,0cyclictest0-21swapper/300:20:003
9725992323,0cyclictest0-21swapper/223:40:032
9725992323,0cyclictest0-21swapper/222:02:122
9725992323,0cyclictest0-21swapper/221:17:012
9725992322,1cyclictest6795-21ssh21:41:162
9725992322,1cyclictest4248-21ssh22:36:332
9725992320,2cyclictest31362-21sh21:33:382
9725992320,0cyclictest0-21swapper/222:20:032
9725992320,0cyclictest0-21swapper/221:48:512
9725992320,0cyclictest0-21swapper/200:26:152
972599230,23cyclictest0-21swapper/221:56:052
972599230,23cyclictest0-21swapper/221:26:412
972599230,23cyclictest0-21swapper/221:23:292
972599230,23cyclictest0-21swapper/221:11:062
972599230,23cyclictest0-21swapper/200:21:452
972599230,22cyclictest6139-21ssh22:08:272
972599230,22cyclictest3752-21diskmemload23:10:292
972599230,22cyclictest22870-21ssh23:22:172
972599230,22cyclictest0-21swapper/222:46:222
972599230,1cyclictest0-21swapper/222:17:542
972599230,0cyclictest0-21swapper/223:50:202
972599230,0cyclictest0-21swapper/222:57:162
972599230,0cyclictest0-21swapper/200:36:122
9721992323,0cyclictest3752-21diskmemload23:14:191
9721992323,0cyclictest3752-21diskmemload22:20:071
9721992323,0cyclictest3752-21diskmemload21:56:591
9721992323,0cyclictest0-21swapper/123:59:461
9721992323,0cyclictest0-21swapper/123:32:041
9721992323,0cyclictest0-21swapper/122:41:221
9721992323,0cyclictest0-21swapper/121:34:091
9721992323,0cyclictest0-21swapper/100:23:541
9721992321,1cyclictest0-21swapper/122:35:001
9721992320,2cyclictest3768-21perf21:10:001
972199230,2cyclictest19799-21ssh23:18:301
972199230,23cyclictest3752-21diskmemload00:25:391
972199230,23cyclictest0-21swapper/122:59:251
972199230,22cyclictest30358-21/usr/sbin/munin22:02:261
972199230,22cyclictest21976-21sh23:49:041
972199230,22cyclictest1268050kworker/u9:000:30:041
972199230,22cyclictest12168-21ssh21:46:001
972199230,22cyclictest0-21swapper/122:51:031
972199230,0cyclictest0-21swapper/123:26:001
972199230,0cyclictest0-21swapper/123:21:531
972199230,0cyclictest0-21swapper/121:24:561
9719992323,0cyclictest3752-21diskmemload21:57:510
9719992323,0cyclictest0-21swapper/021:39:420
971999232,21cyclictest62550irq/127-eno122:12:170
9719992322,1cyclictest62550irq/127-eno121:14:170
9719992322,1cyclictest0-21swapper/023:23:440
9719992321,1cyclictest0-21swapper/023:56:540
9719992321,1cyclictest0-21swapper/023:27:120
971999230,2cyclictest9978-21rm23:38:400
971999230,23cyclictest12547-21kworker/u8:123:46:200
971999230,22cyclictest8617-21cron22:40:000
971999230,22cyclictest3752-21diskmemload23:01:180
971999230,22cyclictest3752-21diskmemload22:31:240
971999230,22cyclictest3752-21diskmemload21:41:350
971999230,22cyclictest3752-21diskmemload00:36:190
971999230,22cyclictest29985-21proc_pri21:32:240
971999230,22cyclictest21777-21sh21:54:030
971999230,22cyclictest0-21swapper/023:05:300
971999230,22cyclictest0-21swapper/021:45:490
971999230,1cyclictest14471-21apt-get19:20:020
971999230,1cyclictest0-21swapper/000:08:010
971999230,0cyclictest0-21swapper/023:54:410
9727992222,0cyclictest3752-21diskmemload22:15:163
9727992222,0cyclictest0-21swapper/323:10:053
9727992222,0cyclictest0-21swapper/322:14:143
9727992221,1cyclictest0-21swapper/322:25:323
9727992220,1cyclictest0-21swapper/322:07:223
9727992220,1cyclictest0-21swapper/300:20:463
972799220,22cyclictest0-21swapper/300:06:233
972799220,21cyclictest9006-21apt-get00:35:023
972799220,21cyclictest7347-21apt-get20:12:143
972799220,21cyclictest12501-21apt-get19:17:143
972799220,21cyclictest10989-21apt-get23:40:023
972799220,21cyclictest0-21swapper/320:10:003
972799220,0cyclictest0-21swapper/323:18:103
972799220,0cyclictest0-21swapper/322:45:443
9725992222,0cyclictest3752-21diskmemload23:49:172
9725992222,0cyclictest3752-21diskmemload21:35:272
9725992221,1cyclictest5661-21ssh23:06:512
9725992220,1cyclictest21760-21apt20:40:012
9725992220,1cyclictest14127-21sh20:25:002
9725992220,0cyclictest0-21swapper/223:35:142
972599220,22cyclictest3752-21diskmemload23:00:412
972599220,21cyclictest3752-21diskmemload23:30:292
972599220,21cyclictest299982chrt22:30:032
972599220,21cyclictest19740-21perf19:30:002
972599220,21cyclictest15990-21ssh23:15:362
972599220,21cyclictest0-21swapper/221:52:182
972599220,1cyclictest25619-21ssh22:26:462
972599220,1cyclictest13442-21ssh00:10:162
972599220,0cyclictest0-21swapper/223:28:392
972599220,0cyclictest0-21swapper/200:00:482
9721992222,0cyclictest0-21swapper/122:11:231
9721992222,0cyclictest0-21swapper/121:50:201
9721992222,0cyclictest0-21swapper/121:17:491
9721992221,1cyclictest6165-21diskmemload23:35:461
9721992221,1cyclictest0-21swapper/100:01:101
9721992221,0cyclictest3462-21apt-get22:07:151
9721992220,1cyclictest937-21ssh21:36:051
9721992220,1cyclictest606-21dbus-daemon20:40:011
9721992220,1cyclictest0-21swapper/121:41:261
9721992220,1cyclictest0-21swapper/120:30:001
972199220,22cyclictest12680-1kworker/u9:023:01:551
972199220,21cyclictest10108-21apt-get19:12:161
972199220,1cyclictest3752-21diskmemload22:47:261
972199220,1cyclictest29759-21apt-get19:52:161
972199220,0cyclictest3752-21diskmemload00:16:081
9719992222,0cyclictest0-21swapper/022:56:030
9719992222,0cyclictest0-21swapper/000:27:200
9719992220,2cyclictest12585-21kworker/u8:122:45:490
9719992220,2cyclictest0-21swapper/022:05:390
971999220,21cyclictest3752-21diskmemload00:19:410
971999220,21cyclictest12089-21awk19:15:010
971999220,21cyclictest0-21swapper/023:19:380
971999220,21cyclictest0-21swapper/022:50:120
971999220,21cyclictest0-21swapper/022:41:370
971999220,21cyclictest0-21swapper/021:15:360
971999220,21cyclictest0-21swapper/021:02:140
971999220,1cyclictest0-21swapper/023:10:310
971999220,1cyclictest0-21swapper/022:22:440
971999220,0cyclictest0-21swapper/000:31:020
9727992120,1cyclictest0-21swapper/322:40:063
9727992119,1cyclictest16520-21apt-get20:30:003
972799210,21cyclictest4882-21ssh23:05:433
972799210,20cyclictest16520-21apt-get20:30:023
972799210,20cyclictest0-21swapper/320:00:023
972799210,20cyclictest0-21swapper/319:32:163
972799210,20cyclictest0-21swapper/319:27:163
972799210,0cyclictest0-21swapper/319:22:163
972599210,20cyclictest0-21swapper/219:22:162
972599210,0cyclictest0-21swapper/220:00:022
972199210,21cyclictest0-21swapper/123:40:111
972199210,20cyclictest0-21swapper/119:32:161
972199210,1cyclictest15113-21apt-get19:20:021
9719992121,0cyclictest0-21swapper/020:17:170
9719992120,0cyclictest0-21swapper/019:37:170
971999210,20cyclictest1720-21apt-get20:00:020
971999210,1cyclictest14475-21perf19:20:010
971999210,0cyclictest22224-21apt-get20:42:160
971999210,0cyclictest0-21swapper/019:57:160
971999210,0cyclictest0-21swapper/019:27:150
9727992018,1cyclictest11737-21turbostat20:20:003
972799200,20cyclictest0-21swapper/319:42:153
972799200,0cyclictest12169-21apt-get20:22:163
9725992020,0cyclictest0-21swapper/220:47:172
972599200,0cyclictest0-21swapper/220:12:152
9721992020,0cyclictest0-21swapper/120:47:181
9721992020,0cyclictest0-21swapper/120:30:021
9721992018,1cyclictest11747-21munin-plugin-st20:20:001
972199200,0cyclictest0-21swapper/119:57:151
9719992020,0cyclictest0-21swapper/020:27:160
9719992018,1cyclictest313-21systemd-journal19:35:000
9719992018,1cyclictest24619-21apt-get20:47:170
971999200,0cyclictest0-21swapper/020:07:160
9727991919,0cyclictest0-21swapper/321:02:143
972799190,0cyclictest0-21swapper/319:47:163
9725991919,0cyclictest0-21swapper/221:07:192
9725991919,0cyclictest0-21swapper/220:32:152
9725991919,0cyclictest0-21swapper/220:07:162
9725991919,0cyclictest0-21swapper/219:17:142
9725991919,0cyclictest0-21swapper/219:12:162
9725991918,1cyclictest22133-21apt19:35:002
9721991919,0cyclictest0-21swapper/119:47:161
9721991918,1cyclictest29408-21apt-get20:57:161
9719991919,0cyclictest0-21swapper/021:07:190
9719991919,0cyclictest0-21swapper/019:47:160
9719991918,1cyclictest14144-21turbostat20:25:000
9719991918,1cyclictest0-21swapper/020:32:160
971999190,1cyclictest7315-21/usr/sbin/munin20:12:150
9725991818,0cyclictest0-21swapper/219:42:152
9721991818,0cyclictest0-21swapper/121:02:151
9721991818,0cyclictest0-21swapper/120:22:161
9719991818,0cyclictest0-21swapper/020:37:160
9727991717,0cyclictest0-21swapper/319:57:163
9727991717,0cyclictest0-21swapper/319:37:173
9727991717,0cyclictest0-21swapper/319:12:163
9725991717,0cyclictest0-21swapper/220:57:162
9725991715,1cyclictest606-21dbus-daemon19:40:002
9721991717,0cyclictest0-21swapper/119:17:141
9719991717,0cyclictest0-21swapper/019:42:150
9727991615,1cyclictest0-21swapper/319:52:163
9727991614,1cyclictest24172-21sh20:45:013
9725991614,1cyclictest26877-21taskset20:50:552
9721991614,1cyclictest0-21swapper/119:27:221
9727991513,1cyclictest0-21swapper/320:37:213
972799150,1cyclictest0-21swapper/320:55:013
9725991515,0cyclictest32155-21apt-get19:57:162
9721991514,1cyclictest22-21ksoftirqd/119:36:381
9721991513,1cyclictest0-21swapper/120:45:011
971999150,14cyclictest0-21swapper/019:54:340
9727991414,0cyclictest0-21swapper/320:47:173
9727991412,1cyclictest31401-21basename21:00:003
972799140,14cyclictest0-21swapper/321:06:233
9725991413,1cyclictest22224-21apt-get20:42:162
972599140,14cyclictest0-21swapper/220:27:432
972599140,0cyclictest30757-21strings19:52:222
972599140,0cyclictest0-21swapper/220:15:462
972599140,0cyclictest0-21swapper/219:47:092
9721991412,1cyclictest4956-21apt-get20:07:161
972199140,0cyclictest0-21swapper/119:40:411
971999140,14cyclictest0-21swapper/020:52:420
9725991313,0cyclictest0-21swapper/221:00:562
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional