You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 12:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Wed Jul 08, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15240993029,1cyclictest13-21ksoftirqd/022:16:230
1524099270,1cyclictest32452-21memory22:56:420
15247992623,2cyclictest13445-21sh22:40:002
15240992624,1cyclictest0-21swapper/023:30:430
1524099260,25cyclictest19426-21sh23:13:330
15251992523,1cyclictest0-21swapper/322:26:423
15251992523,1cyclictest0-21swapper/321:23:123
15247992520,4cyclictest0-21swapper/221:11:242
1524799250,2cyclictest3517-21/usr/sbin/munin22:31:392
1524799250,24cyclictest23350-21rm00:14:252
1524799250,24cyclictest10030-21apt-get20:06:362
1524799250,24cyclictest0-21swapper/221:55:572
15242992524,1cyclictest14046-21rm23:09:261
15242992522,2cyclictest22152-21idleruntime-cro23:45:001
1524299250,1cyclictest20475-21ssh21:48:221
1524099250,24cyclictest8489-21diskmemload22:40:280
1524099250,24cyclictest0-21swapper/021:38:240
15251992422,1cyclictest13439-21cron22:40:003
15251992422,1cyclictest12660-21grep00:05:013
15251992422,1cyclictest0-21swapper/323:36:143
1525199240,2cyclictest9858-21ssh23:34:043
1525199240,23cyclictest8489-21diskmemload23:06:113
1525199240,23cyclictest8489-21diskmemload23:00:443
1525199240,23cyclictest8489-21diskmemload21:28:033
1525199240,23cyclictest18645-21sh23:12:263
1525199240,23cyclictest16124-21apt-get23:40:023
1525199240,23cyclictest0-21swapper/322:53:203
1525199240,0cyclictest0-21swapper/321:55:573
1524799240,24cyclictest8489-21diskmemload00:25:242
1524799240,23cyclictest22160-21dump-pmu-power23:45:002
1524799240,23cyclictest0-21swapper/223:10:542
1524799240,23cyclictest0-21swapper/221:23:122
1524299240,24cyclictest0-21swapper/123:37:131
1524299240,24cyclictest0-21swapper/122:36:571
1524299240,23cyclictest8489-21diskmemload23:26:301
1524299240,0cyclictest8489-21diskmemload00:39:501
1524299240,0cyclictest0-21swapper/122:14:221
15240992423,1cyclictest13-21ksoftirqd/023:39:390
15240992423,1cyclictest0-21swapper/000:15:030
15240992422,1cyclictest0-21swapper/023:28:020
15240992422,1cyclictest0-21swapper/022:21:310
1524099240,24cyclictest8489-21diskmemload23:21:290
1524099240,24cyclictest0-21swapper/021:44:440
1524099240,23cyclictest8489-21diskmemload23:15:580
1524099240,23cyclictest8489-21diskmemload22:39:280
1524099240,23cyclictest31734-21sh21:30:130
1524099240,23cyclictest0-21swapper/023:51:470
1524099240,0cyclictest8489-21diskmemload22:26:220
1524099240,0cyclictest0-21swapper/000:10:260
15251992323,0cyclictest0-21swapper/300:25:243
15251992323,0cyclictest0-21swapper/300:08:273
15251992321,1cyclictest3282-21cron23:00:003
15251992321,1cyclictest26091-21sh23:47:103
15251992321,1cyclictest0-21swapper/323:15:453
15251992321,1cyclictest0-21swapper/321:10:123
15251992320,2cyclictest8489-21diskmemload21:41:493
15251992320,2cyclictest24481-21sh22:21:133
15251992320,2cyclictest1470-21sh22:01:003
15251992320,2cyclictest11770-21awk00:31:473
1525199230,2cyclictest22725-21sh21:51:093
1525199230,23cyclictest8489-21diskmemload23:58:193
1525199230,23cyclictest0-21swapper/323:52:573
1525199230,22cyclictest9843-21sh22:06:473
1525199230,22cyclictest8489-21diskmemload22:32:313
1525199230,0cyclictest0-21swapper/323:20:213
1525199230,0cyclictest0-21swapper/321:37:533
15247992323,0cyclictest0-21swapper/223:36:142
15247992323,0cyclictest0-21swapper/222:41:212
15247992321,1cyclictest31118-21expr00:21:362
15247992321,1cyclictest0-21swapper/222:19:562
15247992320,2cyclictest737-21ssh23:26:382
15247992318,4cyclictest0-21swapper/221:41:122
1524799230,2cyclictest16604-21ssh00:07:522
1524799230,23cyclictest0-21swapper/221:28:262
1524799230,22cyclictest31370-21ssh23:24:312
1524799230,22cyclictest24602-21ssh23:18:002
1524799230,22cyclictest21251-21ssh21:49:292
1524799230,22cyclictest15114-21ssh00:36:042
1524799230,1cyclictest13184-21ssh23:08:082
1524799230,0cyclictest0-21swapper/223:52:572
1524799230,0cyclictest0-21swapper/222:53:202
1524799230,0cyclictest0-21swapper/222:05:162
1524799230,0cyclictest0-21swapper/221:34:412
1524799230,0cyclictest0-21swapper/200:34:242
15242992323,0cyclictest8489-21diskmemload22:25:181
15242992323,0cyclictest0-21swapper/123:54:161
15242992323,0cyclictest0-21swapper/121:18:451
15242992322,1cyclictest8489-21diskmemload00:00:391
15242992321,1cyclictest24913-21sh22:50:121
1524299230,23cyclictest0-21swapper/123:02:531
1524299230,23cyclictest0-21swapper/122:55:221
1524299230,23cyclictest0-21swapper/122:47:431
1524299230,23cyclictest0-21swapper/100:34:351
1524299230,22cyclictest606-21dbus-daemon00:25:011
1524299230,22cyclictest31774-21sh21:30:161
1524299230,22cyclictest30930-21sh23:23:511
1524299230,22cyclictest0-21swapper/123:10:011
1524299230,22cyclictest0-21swapper/121:25:491
1524299230,22cyclictest0-21swapper/100:06:211
1524299230,0cyclictest8489-21diskmemload22:40:551
1524299230,0cyclictest8489-21diskmemload00:10:331
1524299230,0cyclictest0-21swapper/121:42:431
1524299230,0cyclictest0-21swapper/121:10:451
15240992323,0cyclictest62550irq/127-eno121:51:220
15240992323,0cyclictest0-21swapper/023:55:150
15240992323,0cyclictest0-21swapper/021:46:220
15240992322,1cyclictest13-21ksoftirqd/023:06:280
15240992321,1cyclictest8489-21diskmemload22:50:540
15240992321,1cyclictest31191-21sh21:29:510
15240992321,1cyclictest25283-21diskmemload23:46:450
15240992320,2cyclictest13-21ksoftirqd/021:00:010
1524099230,2cyclictest14862-21ssh00:35:440
1524099230,23cyclictest8489-21diskmemload22:10:030
1524099230,23cyclictest0-21swapper/021:56:490
1524099230,22cyclictest855-21ssh00:22:420
1524099230,22cyclictest6200-21ssh00:26:500
1524099230,22cyclictest11882-21rm00:31:490
1524099230,1cyclictest0-21swapper/022:45:030
1524099230,1cyclictest0-21swapper/021:16:260
1524099230,0cyclictest0-21swapper/022:01:110
15251992222,0cyclictest0-21swapper/322:10:133
15251992222,0cyclictest0-21swapper/300:21:193
15251992221,1cyclictest15002-21ssh00:35:553
15251992220,1cyclictest31869-21sh21:30:233
15251992220,1cyclictest29597-21awk19:40:003
1525199220,22cyclictest0-21swapper/300:10:443
1525199220,21cyclictest0-21swapper/322:45:323
1525199220,21cyclictest0-21swapper/322:41:183
1525199220,21cyclictest0-21swapper/300:15:053
1525199220,1cyclictest20361-21smartctl22:16:453
1525199220,1cyclictest0-21swapper/321:45:153
1525199220,0cyclictest0-21swapper/323:27:333
15247992222,0cyclictest8489-21diskmemload00:00:342
15247992222,0cyclictest0-21swapper/221:36:342
15247992221,1cyclictest0-21swapper/222:21:132
15247992221,0cyclictest0-21swapper/221:50:202
15247992220,1cyclictest28876-21latency_hist20:45:012
15247992220,1cyclictest0-21swapper/223:45:042
15247992220,1cyclictest0-21swapper/223:30:012
15247992219,1cyclictest313-21systemd-journal20:40:002
1524799220,22cyclictest0-21swapper/200:16:152
1524799220,21cyclictest29534-21ssh22:25:312
1524799220,21cyclictest1033-21ssh22:00:272
1524799220,21cyclictest0-21swapper/223:55:242
1524799220,21cyclictest0-21swapper/222:45:382
1524799220,21cyclictest0-21swapper/219:56:382
1524799220,1cyclictest0-21swapper/222:55:542
15242992222,0cyclictest0-21swapper/122:07:331
15242992222,0cyclictest0-21swapper/121:56:311
15242992221,1cyclictest8489-21diskmemload23:15:081
15242992221,1cyclictest0-21swapper/120:00:021
15242992220,1cyclictest0-21swapper/122:31:441
1524299220,2cyclictest22408-21turbostat19:25:011
1524299220,22cyclictest0-21swapper/123:58:351
1524299220,21cyclictest8489-21diskmemload22:00:211
1524299220,21cyclictest25182-21apt-get19:31:371
1524299220,21cyclictest23062-21sh21:51:341
1524299220,21cyclictest22786-21apt-get19:26:371
1524299220,21cyclictest0-21swapper/119:40:001
1524299220,1cyclictest313-21systemd-journal21:00:011
1524299220,1cyclictest29734-21ssh00:20:091
1524299220,1cyclictest24573-21ssh22:21:201
1524299220,1cyclictest0-21swapper/122:15:181
1524299220,1cyclictest0-21swapper/121:20:561
1524299220,0cyclictest0-21swapper/123:45:021
1524299220,0cyclictest0-21swapper/100:16:411
15240992222,0cyclictest0-21swapper/022:31:500
15240992220,1cyclictest0-21swapper/022:06:450
15240992219,2cyclictest1235-21wc20:55:010
1524099220,22cyclictest0-21swapper/021:23:120
1524099220,21cyclictest8489-21diskmemload00:00:140
1524099220,21cyclictest0-21swapper/023:00:280
1524099220,21cyclictest0-21swapper/019:21:360
1524099220,1cyclictest0-21swapper/023:40:010
1524099220,1cyclictest0-21swapper/000:06:010
15251992119,1cyclictest28875-21sleep20:45:013
15251992118,2cyclictest26466-21perf20:40:003
1525199210,20cyclictest0-21swapper/319:41:363
1525199210,0cyclictest0-21swapper/320:21:373
15247992120,1cyclictest6277-1kworker/u9:022:10:462
15247992120,0cyclictest12427-21apt-get20:11:372
15247992119,1cyclictest27580-21apt-get19:36:372
1524799210,21cyclictest0-21swapper/223:00:012
1524799210,20cyclictest29994-21apt-get19:41:362
1524799210,1cyclictest15177-21ssh21:16:062
15242992121,0cyclictest0-21swapper/120:51:361
1524299210,0cyclictest0-21swapper/120:21:371
1524299210,0cyclictest0-21swapper/119:41:361
15240992121,0cyclictest0-21swapper/020:26:350
15240992121,0cyclictest0-21swapper/020:21:370
1524099210,21cyclictest0-21swapper/021:06:360
1524099210,21cyclictest0-21swapper/019:26:370
1524099210,0cyclictest0-21swapper/020:36:370
15251992020,0cyclictest0-21swapper/320:11:373
15251992020,0cyclictest0-21swapper/319:56:383
15251992020,0cyclictest0-21swapper/319:26:373
15251992019,0cyclictest0-21swapper/320:01:363
1525199200,19cyclictest1224-21idleruntime-cro20:55:013
15247992020,0cyclictest29245-21apt-get20:46:372
15247992020,0cyclictest0-21swapper/220:01:362
1524799200,0cyclictest17247-21apt-get20:21:372
15242992020,0cyclictest0-21swapper/119:51:351
15242992020,0cyclictest0-21swapper/119:11:351
1524299200,20cyclictest0-21swapper/121:05:011
1524299200,0cyclictest0-21swapper/120:46:371
15240992020,0cyclictest0-21swapper/019:16:390
15240992019,1cyclictest0-21swapper/019:36:370
15251991919,0cyclictest0-21swapper/320:46:373
15247991917,1cyclictest17976-21apt-get19:16:392
1524799190,1cyclictest1226-21sh20:55:012
15242991919,0cyclictest0-21swapper/120:10:011
15242991918,0cyclictest0-21swapper/121:06:361
15240991919,0cyclictest0-21swapper/020:01:360
15240991918,1cyclictest141rcuc/019:41:360
15240991918,1cyclictest13-21ksoftirqd/020:06:540
15251991818,0cyclictest0-21swapper/320:16:383
15251991817,1cyclictest0-21swapper/319:31:373
1525199180,18cyclictest0-21swapper/320:06:373
15247991818,0cyclictest0-21swapper/221:01:372
15247991818,0cyclictest0-21swapper/220:16:382
15247991818,0cyclictest0-21swapper/219:31:372
1524799180,0cyclictest0-21swapper/219:26:372
1524099189,8cyclictest141rcuc/020:46:370
15251991717,0cyclictest0-21swapper/321:01:373
15251991715,1cyclictest21670-21munin-run20:30:003
15247991715,1cyclictest17605-21perf19:15:012
15242991717,0cyclictest0-21swapper/120:11:371
1524299170,10cyclictest0-21swapper/120:26:351
15240991711,6cyclictest13-21ksoftirqd/020:11:380
15251991616,0cyclictest0-21swapper/319:16:393
15251991614,1cyclictest0-21swapper/319:15:013
1525199160,15cyclictest870-21memory19:46:423
1525199160,15cyclictest0-21swapper/319:55:013
15247991615,1cyclictest32390-21apt-get19:46:382
15247991615,0cyclictest2370-21apt-get19:51:362
15242991616,0cyclictest0-21swapper/120:41:371
15242991616,0cyclictest0-21swapper/120:16:391
15240991616,0cyclictest0-21swapper/019:56:370
1524099160,1cyclictest0-21swapper/021:01:410
15251991513,1cyclictest22942-21memory20:31:423
15242991515,0cyclictest24434-21apt-get20:36:371
15240991515,0cyclictest141rcuc/019:11:440
15240991515,0cyclictest0-21swapper/020:41:370
15240991515,0cyclictest0-21swapper/020:16:380
1524099150,1cyclictest23686-21grep20:31:480
1525199140,14cyclictest0-21swapper/320:55:243
1525199140,0cyclictest0-21swapper/319:20:323
15247991414,0cyclictest0-21swapper/221:06:112
15247991412,1cyclictest0-21swapper/220:57:012
1524799140,0cyclictest0-21swapper/220:31:262
1524799140,0cyclictest0-21swapper/220:28:252
1524799140,0cyclictest0-21swapper/219:22:032
15242991414,0cyclictest0-21swapper/119:47:231
1524299140,14cyclictest0-21swapper/119:57:531
1524299140,14cyclictest0-21swapper/119:15:581
1524099140,14cyclictest0-21swapper/019:51:490
1524099140,0cyclictest0-21swapper/019:45:420
1524099140,0cyclictest0-21swapper/019:31:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional