You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 03:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Thu May 28, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1766629481,9sleep00-21swapper/019:08:020
1762329279,9sleep20-21swapper/219:07:282
1752528370,9sleep30-21swapper/319:06:123
1769528265,13sleep10-21swapper/119:08:231
1791099266,18cyclictest0-21swapper/020:25:110
17910992423,1cyclictest0-21swapper/023:48:550
17910992412,2cyclictest13-21ksoftirqd/022:36:390
17914992322,1cyclictest0-21swapper/221:42:422
1791499230,22cyclictest0-21swapper/222:01:332
17913992320,3cyclictest25-21ksoftirqd/120:35:131
1791399230,23cyclictest0-21swapper/100:28:511
1791099239,12cyclictest0-21swapper/000:34:450
1791099238,14cyclictest0-21swapper/023:41:060
17910992323,0cyclictest161rcuc/023:22:390
17914992221,1cyclictest0-21swapper/200:19:522
1791399221,21cyclictest10708-21diskmemload23:52:571
17910992220,2cyclictest0-21swapper/023:27:000
17910992220,1cyclictest10708-21diskmemload23:05:000
17914992118,3cyclictest0-21swapper/223:46:222
17914992117,3cyclictest0-21swapper/222:59:122
1791399211,20cyclictest0-21swapper/120:05:131
17910992118,2cyclictest0-21swapper/022:45:000
17915992012,1cyclictest141rcu_preempt23:16:053
17914992018,1cyclictest0-21swapper/221:12:222
17914992016,4cyclictest0-21swapper/221:45:222
1791499200,12cyclictest0-21swapper/200:21:592
17913992016,3cyclictest25-21ksoftirqd/100:13:341
1791399200,19cyclictest0-21swapper/122:15:011
17910992017,2cyclictest0-21swapper/021:20:250
1791499190,18cyclictest0-21swapper/221:31:322
1791399190,3cyclictest141rcu_preempt00:20:221
1791399190,1cyclictest0-21swapper/122:05:121
1791099194,15cyclictest0-21swapper/000:03:160
1791099194,15cyclictest0-21swapper/000:03:160
1791099191,17cyclictest0-21swapper/023:10:120
17913991818,0cyclictest0-21swapper/122:53:511
17913991817,1cyclictest0-21swapper/122:47:211
17913991814,3cyclictest25-21ksoftirqd/123:15:001
1791399180,1cyclictest0-21swapper/100:31:141
1791099188,9cyclictest0-21swapper/023:37:170
1791099185,7cyclictest5963-21grep21:00:000
17910991816,1cyclictest10708-21diskmemload00:25:360
17910991810,7cyclictest0-21swapper/000:20:130
121432180,0chrt0-21swapper/122:27:171
1791599170,16cyclictest0-21swapper/321:25:143
17914991713,4cyclictest0-21swapper/223:02:022
1791499170,3cyclictest0-21swapper/200:31:522
17913991717,0cyclictest0-21swapper/121:44:331
17913991716,1cyclictest221irq_work/123:19:301
17913991715,2cyclictest25-21ksoftirqd/122:34:551
1791099179,8cyclictest10708-21diskmemload21:57:590
17910991715,2cyclictest10708-21diskmemload22:53:050
17910991715,0cyclictest10708-21diskmemload22:58:460
17910991714,3cyclictest13571-21ssh21:50:120
17910991714,3cyclictest10708-21diskmemload22:13:350
1791599166,4cyclictest7737-21sh22:25:133
17915991612,2cyclictest28372-21cut19:30:173
1791599160,1cyclictest0-21swapper/300:22:263
17914991614,2cyclictest0-21swapper/221:59:222
17914991612,4cyclictest0-21swapper/200:02:222
17914991612,4cyclictest0-21swapper/200:02:222
17914991611,4cyclictest17603-21sensors22:30:222
1791399162,1cyclictest141rcu_preempt23:44:561
17913991615,1cyclictest0-21swapper/123:58:481
17913991615,1cyclictest0-21swapper/100:19:181
1791399161,2cyclictest0-21swapper/122:59:101
1791399160,2cyclictest0-21swapper/121:11:071
1791399160,12cyclictest0-21swapper/121:00:211
1791099169,5cyclictest31075-21/usr/sbin/munin23:55:220
1791099163,13cyclictest0-21swapper/023:07:210
17910991615,1cyclictest10708-21diskmemload22:22:150
17910991615,1cyclictest10708-21diskmemload00:39:130
17910991614,2cyclictest0-21swapper/022:31:140
17910991614,2cyclictest0-21swapper/022:03:110
17910991614,1cyclictest0-21swapper/023:15:130
17910991610,5cyclictest0-21swapper/022:05:120
17915991514,1cyclictest0-21swapper/321:10:363
1791599151,3cyclictest0-21swapper/321:18:293
1791599151,2cyclictest0-21swapper/322:01:383
1791599150,15cyclictest0-21swapper/300:04:113
1791599150,15cyclictest0-21swapper/300:04:113
1791599150,14cyclictest0-21swapper/321:42:503
1791499153,3cyclictest6273-21switchtime19:50:242
1791399153,11cyclictest0-21swapper/122:39:511
17913991512,1cyclictest0-21swapper/123:03:541
1791399150,3cyclictest0-21swapper/121:52:251
1791399150,1cyclictest0-21swapper/123:25:001
1791399150,15cyclictest0-21swapper/123:45:321
1791399150,13cyclictest0-21swapper/100:39:441
1791099158,5cyclictest2921-21rm00:17:160
1791099155,10cyclictest0-21swapper/022:47:240
17910991513,2cyclictest10708-21diskmemload21:43:510
17910991513,2cyclictest0-21swapper/000:08:590
17910991512,2cyclictest21736-21/usr/sbin/munin21:35:190
17910991512,2cyclictest0-21swapper/021:15:210
17910991511,2cyclictest10708-21diskmemload21:32:030
1791099150,1cyclictest13-21ksoftirqd/020:00:130
1791599144,10cyclictest0-21swapper/322:40:133
17915991413,1cyclictest22550-21ssh00:10:133
1791599141,1cyclictest10708-21diskmemload23:52:123
17915991411,2cyclictest19190-21ssh22:50:253
1791599140,12cyclictest0-21swapper/321:37:513
1791599140,0cyclictest0-21swapper/321:55:333
1791499149,4cyclictest10708-21diskmemload21:16:302
17914991411,3cyclictest0-21swapper/223:41:522
17914991411,3cyclictest0-21swapper/200:27:222
17914991411,2cyclictest15679-21cat22:30:012
17914991410,2cyclictest15262-21grep00:05:232
1791499140,0cyclictest0-21swapper/223:06:252
1791399141,4cyclictest225-21systemd-journal20:28:141
17913991410,2cyclictest11992-21wc21:30:011
1791399140,2cyclictest0-21swapper/122:41:361
1791399140,14cyclictest0-21swapper/121:32:551
1791099149,2cyclictest161rcuc/000:13:420
1791099147,6cyclictest0-21swapper/022:15:440
17910991412,1cyclictest14985-21grep20:10:160
17910991412,1cyclictest10708-21diskmemload21:47:440
17910991411,2cyclictest8819-21sed19:55:240
17910991411,2cyclictest10034-21cat21:05:240
17910991411,2cyclictest10034-21cat21:05:240
17910991410,3cyclictest14085-21basename20:10:000
17910991410,3cyclictest10708-21diskmemload21:25:020
1791099140,8cyclictest21242-21awk20:25:000
1791599139,3cyclictest7934-21grep21:00:273
1791599137,5cyclictest15648-21grep00:25:143
1791599132,6cyclictest225-21systemd-journal20:40:003
17915991310,2cyclictest5292-21grep23:40:173
17915991310,2cyclictest3834-21grep23:20:153
17915991310,2cyclictest15740-21sed00:05:283
17915991310,2cyclictest14170-21grep21:30:263
1791599130,12cyclictest23740-21head19:20:163
1791599130,11cyclictest5594-21sed19:50:153
1791499139,2cyclictest31669-21ls19:35:262
17914991310,2cyclictest7480-21grep19:55:132
17914991310,2cyclictest5484-21tail20:55:252
17914991310,2cyclictest4098-21grep19:45:252
17914991310,2cyclictest19375-21head19:10:212
17914991310,2cyclictest12716-21tr23:25:192
1791399139,3cyclictest11493-21grep23:25:131
1791399139,2cyclictest26178-21wc20:35:001
1791399138,1cyclictest141rcu_preempt19:55:151
17913991311,1cyclictest25-21ksoftirqd/122:23:381
17913991310,2cyclictest29873-21ls22:00:001
17913991310,2cyclictest20371-21wc20:20:211
17913991310,2cyclictest19109-21chrt22:12:061
1791099139,3cyclictest19126-21cat19:10:190
1791099139,3cyclictest0-21swapper/023:34:480
1791099139,1cyclictest0-21swapper/023:52:020
17910991312,1cyclictest13-21ksoftirqd/021:14:370
17910991312,1cyclictest10708-21diskmemload22:25:190
1791599129,2cyclictest5917-21grep21:45:163
1791599129,2cyclictest32351-21tr00:35:153
1791599129,2cyclictest29487-21rm22:18:383
1791599129,2cyclictest26094-21grep19:25:173
1791599129,2cyclictest21686-21tr20:25:123
1791599129,2cyclictest19790-21sed19:10:263
1791599129,2cyclictest17451-21head20:15:163
1791599129,2cyclictest1242-21wc23:00:003
1791599129,2cyclictest11696-21idleruntime-cro20:05:003
1791599125,5cyclictest7197-21apt-get19:55:123
1791599123,7cyclictest30968-21apache_processe00:15:123
1791599122,9cyclictest0-21swapper/322:36:253
17915991211,1cyclictest0-21swapper/322:08:213
17915991211,1cyclictest0-21swapper/320:20:123
17915991210,1cyclictest19852-21grep23:10:193
17915991210,1cyclictest15663-21tune2fs22:10:153
1791499129,2cyclictest9559-21sort22:45:162
1791499129,2cyclictest31246-21sh22:20:002
1791499129,2cyclictest26193-21apache200:39:222
1791499129,2cyclictest25250-21grep22:35:162
1791499129,2cyclictest20689-21ssh21:35:132
1791499129,2cyclictest18318-21cat22:50:182
1791499129,2cyclictest1655-21grep22:40:192
1791499129,2cyclictest12689-21grep20:05:162
1791499128,3cyclictest26022-21cron20:35:002
1791499128,3cyclictest19739-21wc23:10:182
1791499128,3cyclictest1011-21cat20:50:002
1791499124,5cyclictest29072-21cron23:55:002
17914991210,1cyclictest0-21swapper/223:20:362
1791499120,1cyclictest0-21swapper/223:33:262
1791399129,2cyclictest6097-21grep00:00:171
1791399129,2cyclictest6097-21grep00:00:171
1791399129,2cyclictest6061-21grep22:05:001
1791399129,2cyclictest5526-21cut21:45:141
1791399129,2cyclictest4528-21head20:55:161
1791399129,2cyclictest31662-21grep20:45:161
1791399129,2cyclictest29406-21sed21:20:201
1791399129,2cyclictest24659-21grep19:20:251
1791399129,2cyclictest14449-21sort00:05:171
1791399128,1cyclictest28713-21cut20:40:131
1791399127,2cyclictest32478-21ssh23:36:591
1791399124,4cyclictest225-21systemd-journal19:35:001
1791399121,11cyclictest10708-21diskmemload23:31:481
1791099129,2cyclictest737-21sort19:40:160
1791099128,4cyclictest0-21swapper/020:45:130
1791099128,3cyclictest21803-21grep19:15:230
1791099126,5cyclictest0-21swapper/020:55:000
17910991211,1cyclictest13-21ksoftirqd/019:24:550
1791599119,1cyclictest20413-21sed23:30:143
1791599119,1cyclictest1337-21tr20:50:143
1791599118,2cyclictest3684-21rm22:22:153
1791599118,2cyclictest3310-21sh23:58:473
1791599118,2cyclictest3240-21ls23:00:213
1791599118,2cyclictest3167-21head19:45:163
1791599118,2cyclictest31290-21tr20:45:133
1791599118,2cyclictest27605-21ssh00:31:563
1791599118,2cyclictest2672-21sh23:39:033
1791599118,2cyclictest1948-21taskset19:42:113
1791599118,2cyclictest15789-21ssh22:30:023
1791599118,2cyclictest15371-21ssh21:50:243
1791599118,2cyclictest12607-21ssh23:45:053
1791599118,2cyclictest12232-21grep23:25:163
1791599118,2cyclictest10715-21expr23:05:143
1791599113,7cyclictest15751-21munin-node20:10:233
1791499118,2cyclictest8477-21tail22:05:262
1791499118,2cyclictest4052-21ssh21:25:122
1791499118,2cyclictest29663-21ssh23:55:142
1791499118,2cyclictest28807-21cut20:40:142
1791499118,2cyclictest28726-21cut21:20:152
1791499118,2cyclictest28286-21ssh23:15:192
1791499118,2cyclictest28213-21ssh23:35:022
1791499118,2cyclictest27825-21grep19:30:142
1791499118,2cyclictest23497-21cat00:10:182
1791499118,2cyclictest20795-21unixbench_singl20:20:262
1791499118,2cyclictest1751-21ssh22:20:442
1791499118,2cyclictest14477-21sort21:50:162
1791499118,2cyclictest14400-21hostname20:10:142
1791499118,2cyclictest10238-21sort20:00:162
1791499111,6cyclictest225-21systemd-journal19:15:212
17914991111,0cyclictest0-21swapper/220:50:132
1791499110,11cyclictest0-21swapper/222:10:152
1791399118,2cyclictest9081-21grep21:05:141
1791399118,2cyclictest9081-21grep21:05:141
1791399118,2cyclictest31543-21wc19:35:261
1791399118,2cyclictest28633-21tr21:40:011
1791399118,2cyclictest26473-21tr19:25:191
1791399118,2cyclictest18917-21cut19:10:171
1791399118,2cyclictest1883-21cut20:50:161
1791399118,2cyclictest14825-21df20:10:151
1791399118,2cyclictest12523-21ssh23:05:381
1791399117,3cyclictest21478-21cat19:15:191
1791399110,10cyclictest6234-21grep19:50:241
1791399110,10cyclictest24634-21sh21:17:461
1791099118,2cyclictest7350-21tr21:00:210
1791099118,2cyclictest4886-21tr19:50:120
1791099118,2cyclictest29204-21cat19:30:260
1791099118,2cyclictest28924-21tr20:40:140
1791099118,2cyclictest16933-21tr20:15:130
1791599108,1cyclictest8696-21ssh22:45:083
1791599108,1cyclictest4614-21smartctl20:55:163
1791599108,1cyclictest28058-21cut21:20:123
1791599108,1cyclictest23932-21basename20:30:143
1791599108,1cyclictest20794-21cut19:15:153
1791599107,2cyclictest29034-21expr20:40:143
1791599107,2cyclictest12089-21basename20:05:143
1791599107,2cyclictest10709-21awk21:10:003
1791599107,2cyclictest10709-21awk21:10:003
1791499108,1cyclictest80372sleep221:01:272
1791499108,1cyclictest466-21expr19:40:142
1791499108,1cyclictest22629-21cat20:25:172
1791499107,2cyclictest27172-21if_enp2s020:35:182
1791499107,2cyclictest26691-21tr19:25:212
1791499107,2cyclictest23619-21fschecks_time19:20:152
1791499102,4cyclictest225-21systemd-journal21:06:042
1791499102,4cyclictest225-21systemd-journal21:06:042
1791399108,1cyclictest4435-21taskset19:48:391
1791399108,1cyclictest185032sleep120:15:581
1791399108,1cyclictest10899-21grep20:00:221
1791399106,3cyclictest303-21grep19:40:141
1791099108,1cyclictest2465-21taskset19:45:130
1791099107,2cyclictest30205-21tr19:35:140
1791099107,2cyclictest27233-21uname20:35:180
1791099107,2cyclictest25156-21head20:30:220
1791099105,4cyclictest0-21swapper/019:25:140
179159997,1cyclictest318522sleep319:35:373
179149997,1cyclictest185192sleep220:16:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional