You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 07:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Jul 28, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17975210275,9sleep20-21swapper/219:05:512
1795629784,9sleep00-21swapper/019:05:380
1797629249,39sleep30-21swapper/319:05:523
1811028164,13sleep10-21swapper/119:07:351
18388992724,2cyclictest11488-21apache_processe23:25:141
1839099250,23cyclictest0-21swapper/221:50:312
1839099250,18cyclictest0-21swapper/223:10:472
1839099230,22cyclictest0-21swapper/223:39:452
18388992323,0cyclictest0-21swapper/121:15:191
18388992320,3cyclictest0-21swapper/121:42:351
1839499221,20cyclictest11637-21kworker/3:121:09:013
1839099220,22cyclictest0-21swapper/221:33:092
1839499211,19cyclictest11637-21kworker/3:121:10:063
18390992118,2cyclictest11618-21diskmemload00:10:122
1839099210,21cyclictest0-21swapper/222:53:382
1838899210,0cyclictest0-21swapper/122:07:301
18384992120,1cyclictest0-21swapper/000:00:250
18384992120,1cyclictest0-21swapper/000:00:240
18394992018,2cyclictest0-21swapper/319:10:033
18390992018,1cyclictest25765-21ssh21:56:032
1838899206,8cyclictest11618-21diskmemload22:40:131
18388992016,3cyclictest31806-21turbostat.cron22:24:551
1838899200,2cyclictest0-21swapper/123:58:351
1838899200,1cyclictest0-21swapper/122:19:411
1839099190,17cyclictest0-21swapper/223:30:142
1838899190,1cyclictest0-21swapper/123:35:131
1838899190,18cyclictest0-21swapper/100:10:191
18384991918,1cyclictest0-21swapper/021:29:460
1838499190,19cyclictest0-21swapper/020:30:470
18390991817,1cyclictest0-21swapper/223:05:442
18390991817,1cyclictest0-21swapper/222:45:032
18390991817,1cyclictest0-21swapper/200:19:452
18390991814,2cyclictest12173-21kworker/u8:221:15:282
1839099181,17cyclictest0-21swapper/223:23:412
1839099180,18cyclictest0-21swapper/223:17:322
1839099180,18cyclictest0-21swapper/222:29:322
1839099180,18cyclictest0-21swapper/200:34:222
1839099180,18cyclictest0-21swapper/200:26:232
1839099180,17cyclictest0-21swapper/222:01:532
1838899182,13cyclictest11618-21diskmemload23:41:201
1838899181,3cyclictest18652-21kworker/1:100:33:071
1838499186,1cyclictest141rcu_preempt23:21:360
1838499180,0cyclictest0-21swapper/021:50:300
18394991714,1cyclictest0-21swapper/322:26:023
18394991710,2cyclictest26597-21latency_hist23:15:003
18390991717,0cyclictest0-21swapper/222:55:532
1839099171,3cyclictest141rcu_preempt22:16:312
1839099170,1cyclictest0-21swapper/200:22:512
1839099170,17cyclictest0-21swapper/222:11:012
1839099170,17cyclictest0-21swapper/222:11:002
1839099170,0cyclictest0-21swapper/221:40:532
18388991715,1cyclictest11618-21diskmemload22:10:131
18388991715,1cyclictest11618-21diskmemload22:10:121
18388991713,2cyclictest0-21swapper/121:13:371
1838899170,1cyclictest0-21swapper/123:51:101
299522160,0sleep30-21swapper/300:33:543
18390991616,0cyclictest0-21swapper/221:45:522
1839099160,16cyclictest0-21swapper/221:21:552
1839099160,15cyclictest0-21swapper/221:25:102
1839099160,15cyclictest0-21swapper/221:00:192
1838899161,1cyclictest11556-21ssh21:28:451
1838899160,1cyclictest0-21swapper/100:39:471
1838899160,16cyclictest0-21swapper/121:21:341
18384991610,6cyclictest5388-21apache222:31:460
18390991514,1cyclictest0-21swapper/222:32:332
1839099151,2cyclictest141rcu_preempt23:01:202
1839099150,15cyclictest0-21swapper/221:39:342
1838899150,2cyclictest0-21swapper/122:47:091
1838899150,1cyclictest0-21swapper/122:53:381
1838499157,7cyclictest9997-21munin-run19:59:590
1838499156,6cyclictest1-21systemd19:30:030
1838499152,1cyclictest0-21swapper/022:18:090
18384991513,2cyclictest0-21swapper/022:24:170
1838499151,11cyclictest11997-21munin-plugin-st23:45:000
1838499150,9cyclictest410-21tr19:40:000
1839499149,3cyclictest3964-21wc23:20:193
1839499145,7cyclictest13258-21fschecks_time23:45:183
1839499141,6cyclictest225-21systemd-journal00:30:013
18394991410,3cyclictest30948-21wc23:55:283
18394991410,3cyclictest2839-21cut20:50:183
18394991410,3cyclictest27409-21tr20:35:163
18388991410,1cyclictest0-21swapper/100:25:061
1838899140,1cyclictest0-21swapper/121:34:431
1838499149,2cyclictest12373-21ssh21:10:130
1838499148,3cyclictest28688-21cron23:55:000
1838499140,8cyclictest19779-21sendmail-msp20:20:000
1839499139,3cyclictest8251-21sed21:45:243
1839499139,3cyclictest6274-21ssh00:20:013
1839499139,3cyclictest2328-21cron23:19:593
1839499139,3cyclictest15789-21wc21:50:203
1839499138,4cyclictest21515-21cron21:35:003
18394991311,1cyclictest0-21swapper/322:06:223
18394991310,2cyclictest6772-21ssh00:20:143
18394991310,2cyclictest29461-21head19:30:193
18394991310,2cyclictest27676-21grep19:25:263
18394991310,2cyclictest21214-21wc23:50:143
18394991310,2cyclictest18234-21cut20:15:173
18394991310,2cyclictest14888-21wc00:05:213
18394991310,2cyclictest10005-21perf20:00:003
1839099139,1cyclictest30037-21grep23:55:182
1839099134,3cyclictest0-21swapper/223:27:452
18390991312,1cyclictest0-21swapper/219:35:142
18390991310,2cyclictest32469-21grep22:20:142
1839099130,4cyclictest0-21swapper/200:08:182
1839099130,1cyclictest0-21swapper/223:42:522
1839099130,13cyclictest0-21swapper/221:14:042
1839099130,12cyclictest15131-21cron22:10:012
1838899138,5cyclictest38150irq/131-snd_hda23:02:371
1838899135,3cyclictest11618-21diskmemload21:10:001
1838899132,10cyclictest11618-21diskmemload23:34:521
18388991312,1cyclictest25-21ksoftirqd/122:02:421
18388991311,1cyclictest0-21swapper/100:15:031
18388991310,2cyclictest4110-21ls19:45:191
18388991310,2cyclictest28067-21cat23:15:201
18388991310,2cyclictest17862-21grep20:15:141
1838899130,2cyclictest0-21swapper/122:37:071
1838899130,2cyclictest0-21swapper/121:46:471
1838499133,2cyclictest0-21swapper/000:19:340
1838499131,1cyclictest0-21swapper/021:55:070
18384991310,2cyclictest9689-21grep22:25:230
18384991310,2cyclictest2722-21fschecks_time20:50:170
18384991310,2cyclictest25494-21cron19:24:590
18384991310,2cyclictest15129-21ssh22:10:010
1838499130,2cyclictest0-21swapper/000:22:430
1838499130,1cyclictest0-21swapper/021:40:530
1838499130,1cyclictest0-21swapper/021:31:520
1839499129,2cyclictest7211-21ssh00:00:273
1839499129,2cyclictest7211-21ssh00:00:263
1839499129,2cyclictest31178-21sed21:20:253
1839499129,2cyclictest30645-21cut20:40:203
1839499129,2cyclictest26513-21cut22:35:233
1839499129,2cyclictest24966-21grep21:55:293
1839499129,2cyclictest1443-21head00:35:273
1839499129,2cyclictest12925-21grep20:05:153
1839499129,2cyclictest11050-21cut20:00:183
1839499128,3cyclictest32328-21grep21:40:263
1839499128,3cyclictest23355-21wc20:25:203
1839499128,3cyclictest1399-21grep22:20:223
1839499126,5cyclictest5388-21apache221:25:183
1839499124,7cyclictest0-21swapper/321:17:553
1839499121,8cyclictest17497-21fschecks_count22:30:183
1839499121,5cyclictest0-21swapper/321:35:203
18394991210,1cyclictest32538-21sort20:45:173
18394991210,1cyclictest30083-21sed23:35:283
1839499120,1cyclictest0-21swapper/323:34:233
1839099129,2cyclictest6075-21head00:00:182
1839099129,2cyclictest6075-21head00:00:172
1839099129,2cyclictest25510-21basename19:25:002
1839099129,2cyclictest22661-21tr20:25:162
1839099129,2cyclictest21110-21grep20:20:212
1839099129,2cyclictest21110-21grep20:20:202
1839099129,2cyclictest19728-21cron20:20:002
1839099128,3cyclictest9745-21chrt19:56:572
1839099125,2cyclictest15249-21apt-get20:10:152
1839099123,8cyclictest0-21swapper/220:50:012
1839099123,3cyclictest11618-21diskmemload00:36:052
1839099121,10cyclictest20230-21sed19:10:272
1838899129,2cyclictest8594-21grep19:55:181
1838899129,2cyclictest22264-21ssh21:35:141
1838899129,2cyclictest14637-21grep00:05:191
1838899128,3cyclictest20714-21cron19:15:001
1838899127,2cyclictest27954-21ssh22:55:281
1838899124,6cyclictest11618-21diskmemload23:21:441
1838899121,2cyclictest12750-21kworker/1:123:08:231
1838899120,2cyclictest0-21swapper/122:29:521
1838899120,1cyclictest0-21swapper/100:00:151
1838899120,1cyclictest0-21swapper/100:00:141
1838499129,3cyclictest30803-21munin-node22:38:470
1838499129,2cyclictest9780-21ls23:05:000
1838499129,2cyclictest5391-21ls19:50:000
1838499129,2cyclictest30282-21grep20:40:170
1838499129,2cyclictest28971-21ssh23:35:190
1838499129,2cyclictest28510-21cron21:20:000
1838499129,2cyclictest28223-21ssh21:38:430
1838499129,2cyclictest20691-21grep23:30:180
1838499129,2cyclictest18433-21grep22:50:170
1838499129,2cyclictest15901-21grep00:25:180
1838499129,2cyclictest14644-21grep00:05:190
1838499129,2cyclictest11798-21cut23:25:170
1838499128,1cyclictest20522-21sed23:10:280
1838499125,2cyclictest0-21swapper/000:14:460
18384991210,1cyclictest26204-21grep19:25:150
18384991210,1cyclictest22638-21grep19:15:570
1838499120,1cyclictest0-21swapper/023:18:350
1838499120,11cyclictest0-21swapper/020:00:140
1839499118,2cyclictest9301-21wc21:05:003
1839499118,2cyclictest6779-21ssh22:43:193
1839499118,2cyclictest6684-21ssh23:02:353
1839499118,2cyclictest5368-21apache222:53:033
1839499118,2cyclictest5283-21grep23:40:193
1839499118,2cyclictest32150-21grep22:00:183
1839499118,2cyclictest30439-21chrt22:57:213
1839499118,2cyclictest28896-21ssh22:17:433
1839499118,2cyclictest20808-21head20:20:183
1839499118,2cyclictest20808-21head20:20:173
1839499118,2cyclictest15412-21ssh22:10:033
1839499118,2cyclictest15412-21ssh22:10:023
1839499118,2cyclictest12323-21head23:25:193
1839499118,2cyclictest10843-21tr22:45:233
1839499117,3cyclictest10599-21cut23:05:153
1839499116,2cyclictest225-21systemd-journal19:44:593
1839499112,4cyclictest22289-21ssh00:10:143
1839099119,1cyclictest0-21swapper/223:51:352
1839099118,2cyclictest9637-21grep21:05:162
1839099118,2cyclictest6224-21grep19:50:162
1839099118,2cyclictest28792-21grep20:35:272
1839099118,2cyclictest27607-21sh22:35:492
1839099118,2cyclictest26418-21expr19:25:152
1839099118,2cyclictest15506-21sh23:46:142
1839099118,2cyclictest13974-21wc20:05:232
1839099118,2cyclictest1193-21grep19:40:162
1839099118,2cyclictest11352-21cat20:00:222
1839099117,3cyclictest4981-21seq20:55:152
1839099112,8cyclictest1-21systemd19:30:222
1839099111,9cyclictest0-21swapper/220:45:142
1839099110,11cyclictest3283-21chrt22:40:362
1839099110,11cyclictest0-21swapper/220:30:162
1839099110,10cyclictest4951-21chrt19:47:072
1838899119,1cyclictest15480-21head21:50:181
1838899118,2cyclictest9881-21ssh00:21:201
1838899118,2cyclictest789-21basename19:40:161
1838899118,2cyclictest6649-21tr19:50:201
1838899118,2cyclictest21730-21cut19:15:481
1838899118,2cyclictest19527-21hostname23:10:191
1838899118,2cyclictest17162-21cat22:30:151
1838899118,2cyclictest13560-21chrt23:45:201
1838899116,2cyclictest225-21systemd-journal20:55:011
1838899114,3cyclictest225-21systemd-journal20:25:011
1838899114,3cyclictest225-21systemd-journal20:25:001
1838899110,1cyclictest0-21swapper/121:55:351
1838499119,1cyclictest705-21basename19:40:150
1838499119,1cyclictest6789-21expr21:45:130
1838499119,1cyclictest5495-21ssh22:42:180
1838499119,1cyclictest32707-21smartctl20:45:190
1838499119,1cyclictest32005-21ssh22:00:160
1838499119,1cyclictest30111-21ssh21:20:170
1838499119,1cyclictest28975-21rm23:55:020
1838499119,1cyclictest23427-21seq00:30:110
1838499119,1cyclictest16340-21cat20:10:210
1838499119,1cyclictest153532sleep023:46:060
1838499119,1cyclictest1474-21ssh00:35:270
1838499119,1cyclictest12628-21basename20:05:110
1838499119,1cyclictest12264-21sed23:05:260
1838499118,2cyclictest9721-21grep21:05:170
1838499118,2cyclictest6741-21grep19:50:220
1838499118,2cyclictest29063-21chrt20:37:450
1838499118,2cyclictest25872-21ssh22:55:060
1838499118,2cyclictest10128-21uniq22:45:170
1838499112,8cyclictest0-21swapper/022:10:050
1838499112,8cyclictest0-21swapper/022:10:050
1839499108,1cyclictest24850-21cut20:30:133
1839499108,1cyclictest23386-21apache_accesses19:20:113
1839499108,1cyclictest16768-21cat20:10:273
1839499107,2cyclictest6448-21tr19:50:183
1839499107,2cyclictest32669-21taskset19:37:583
1839499107,2cyclictest22644-21sed19:15:563
1839499100,9cyclictest6618-21taskset20:57:223
1839099107,2cyclictest21280-21tr19:15:452
1839099105,2cyclictest225-21systemd-journal20:42:462
1838899109,1cyclictest0-21swapper/120:10:151
1838899108,2cyclictest25049-21expr20:30:151
1838899108,1cyclictest31879-21taskset20:45:101
1838899108,1cyclictest30703-21basename19:35:121
1838899108,1cyclictest28481-21chrt19:30:141
1838899108,1cyclictest26243-21cut19:25:151
1838899108,1cyclictest11055-21grep20:00:191
1838899107,2cyclictest8669-21cat21:00:281
1838899107,2cyclictest29937-21cat20:40:151
1838899107,2cyclictest22829-21cat20:25:161
1838899100,6cyclictest29065-21sshd20:37:451
1838499109,1cyclictest9350-21date21:04:590
1838499108,1cyclictest24398-21taskset20:28:470
1838499108,1cyclictest20120-21hostname20:20:150
1838499108,1cyclictest20120-21hostname20:20:150
1838499100,9cyclictest4688-21aten2_r7power_c20:55:150
183949997,1cyclictest3608-21sed19:45:153
183889997,2cyclictest5475-21cut20:55:191
183889994,4cyclictest38150irq/131-snd_hda20:09:431
183849998,1cyclictest20673-21chrt19:14:310
183889987,1cyclictest251382sleep119:20:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional