You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 17:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat Apr 25, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1824929986,8sleep00-21swapper/007:05:530
1764029778,14sleep30-21swapper/307:05:243
1834928362,6sleep10-21swapper/107:07:111
1642828270,8sleep20-21swapper/207:05:032
261722600,0sleep00-21swapper/011:33:510
229422450,0sleep00-21swapper/011:12:290
1865899330,32cyclictest0-21swapper/212:04:552
1865899300,29cyclictest0-21swapper/212:19:072
1866399291,27cyclictest12636-21kworker/3:209:18:523
1865899279,17cyclictest11388-21diskmemload10:19:242
1865599271,23cyclictest14159-21seq09:50:021
1865599270,1cyclictest0-21swapper/107:25:171
18663992624,2cyclictest3810-21kworker/3:011:04:433
18658992610,15cyclictest11388-21diskmemload10:12:412
1865899260,25cyclictest0-21swapper/209:45:082
1865899260,25cyclictest0-21swapper/209:30:142
1865899260,22cyclictest0-21swapper/210:52:542
18663992522,3cyclictest3810-21kworker/3:011:33:333
1866399250,24cyclictest12636-21kworker/3:209:37:533
1865899253,21cyclictest11388-21diskmemload11:29:502
1865599250,4cyclictest0-21swapper/110:04:011
1865599250,4cyclictest0-21swapper/110:04:011
1865599250,2cyclictest0-21swapper/110:07:381
1865599250,1cyclictest0-21swapper/109:27:301
1865399250,25cyclictest0-21swapper/009:22:040
18663992422,2cyclictest3810-21kworker/3:011:20:233
1865899246,12cyclictest0-21swapper/210:20:132
1865899241,19cyclictest11388-21diskmemload11:21:382
1865399240,1cyclictest0-21swapper/010:40:370
18663992321,2cyclictest11835-21kworker/3:112:27:333
1865899236,16cyclictest11388-21diskmemload09:28:362
1865899232,20cyclictest22306-21fschecks_count09:35:172
1865899230,23cyclictest0-21swapper/212:20:372
1865899230,22cyclictest11388-21diskmemload11:35:322
1865899230,21cyclictest0-21swapper/209:11:062
1865599239,14cyclictest0-21swapper/112:00:131
18655992315,4cyclictest25-21ksoftirqd/109:21:491
1865599230,2cyclictest0-21swapper/109:35:271
18663992219,2cyclictest11835-21kworker/3:112:18:033
1866399220,21cyclictest11835-21kworker/3:112:11:433
1865899222,19cyclictest11388-21diskmemload10:43:062
1865899221,21cyclictest11388-21diskmemload11:56:272
1865899221,19cyclictest0-21swapper/209:57:452
1865899220,21cyclictest0-21swapper/211:52:122
18655992211,3cyclictest25-21ksoftirqd/110:28:291
18663992119,2cyclictest3810-21kworker/3:011:08:533
18663992119,2cyclictest12636-21kworker/3:209:55:123
18663992118,3cyclictest3810-21kworker/3:011:10:033
18663992118,2cyclictest3810-21kworker/3:010:50:433
1866399210,20cyclictest3810-21kworker/3:011:16:433
1865899215,15cyclictest55650irq/127-enp2s0-11:49:362
1865899212,16cyclictest11388-21diskmemload11:34:022
1865899211,19cyclictest11388-21diskmemload10:36:462
18658992110,10cyclictest55650irq/127-enp2s0-10:55:142
1865899210,20cyclictest0-21swapper/211:18:052
1865899210,19cyclictest0-21swapper/209:00:262
1865599210,1cyclictest0-21swapper/112:10:011
18663992018,2cyclictest3810-21kworker/3:011:59:233
18663992018,2cyclictest3810-21kworker/3:011:46:233
18663992018,2cyclictest3810-21kworker/3:010:39:233
18663992018,2cyclictest12636-21kworker/3:209:26:033
18663992017,3cyclictest3810-21kworker/3:010:13:533
1865899205,15cyclictest11388-21diskmemload12:06:582
1865899201,18cyclictest11388-21diskmemload09:19:212
1865899200,20cyclictest0-21swapper/209:40:082
1865899200,19cyclictest0-21swapper/212:30:002
1865899200,19cyclictest0-21swapper/207:35:172
1865899200,18cyclictest22607-21fschecks_time12:10:172
1865899200,15cyclictest0-21swapper/211:05:462
1865899200,13cyclictest0-21swapper/210:45:052
1865599203,6cyclictest25-21ksoftirqd/109:18:551
1865599200,1cyclictest0-21swapper/111:56:171
18663991918,1cyclictest3810-21kworker/3:010:15:533
18663991917,2cyclictest3810-21kworker/3:010:03:433
18663991917,2cyclictest3810-21kworker/3:010:03:433
18663991917,2cyclictest12636-21kworker/3:209:22:233
18663991917,2cyclictest12636-21kworker/3:209:13:033
1865899190,18cyclictest0-21swapper/212:37:122
1865899190,16cyclictest0-21swapper/210:02:002
1865899190,16cyclictest0-21swapper/210:02:002
1865899190,10cyclictest0-21swapper/211:13:252
18655991916,2cyclictest241rcuc/110:46:401
1865599190,4cyclictest0-21swapper/111:42:071
1865599190,1cyclictest141rcu_preempt11:04:051
1865599190,1cyclictest0-21swapper/112:23:571
1865399197,6cyclictest24807-21munin-node09:15:100
18663991816,2cyclictest12636-21kworker/3:209:54:433
18663991816,2cyclictest12636-21kworker/3:209:46:333
18663991816,2cyclictest11835-21kworker/3:112:31:333
1866399180,1cyclictest0-21swapper/309:41:133
1865899182,15cyclictest0-21swapper/208:25:012
1865899181,17cyclictest11388-21diskmemload09:24:182
1865899181,13cyclictest11388-21diskmemload10:25:282
1865899180,8cyclictest0-21swapper/211:03:272
1865899180,17cyclictest0-21swapper/211:44:052
1865899180,17cyclictest0-21swapper/208:15:142
1865899180,17cyclictest0-21swapper/208:05:152
1865899180,16cyclictest11388-21diskmemload10:30:162
1865599184,2cyclictest25-21ksoftirqd/111:52:471
18655991817,1cyclictest0-21swapper/112:39:551
1865399185,7cyclictest12813-21munin-run08:05:000
1866399172,12cyclictest10704-21apt-get08:00:143
1866399170,2cyclictest3810-21kworker/3:011:41:193
1865899175,10cyclictest6121-21sed08:55:282
18658991711,5cyclictest55650irq/127-enp2s0-10:05:312
1865899170,17cyclictest0-21swapper/212:32:592
1865899170,17cyclictest0-21swapper/209:05:192
1865899170,16cyclictest0-21swapper/208:45:152
1865599172,1cyclictest141rcu_preempt09:13:191
18655991712,4cyclictest11388-21diskmemload10:23:281
1865599171,0cyclictest141rcu_preempt09:32:341
18655991710,5cyclictest241rcuc/111:25:001
1865399170,0cyclictest0-21swapper/010:29:260
1866399160,4cyclictest0-21swapper/312:00:133
1866399160,16cyclictest0-21swapper/310:28:133
1865899166,10cyclictest0-21swapper/209:51:432
1865899161,14cyclictest0-21swapper/207:50:232
1865899161,13cyclictest0-21swapper/208:50:212
1865899160,15cyclictest0-21swapper/207:10:292
1865899160,12cyclictest0-21swapper/207:45:152
1865599163,1cyclictest141rcu_preempt09:47:461
1865599162,1cyclictest141rcu_preempt12:18:021
18655991610,5cyclictest241rcuc/111:45:111
1865599160,1cyclictest0-21swapper/110:42:011
18653991616,0cyclictest0-21swapper/010:22:280
1865399160,1cyclictest0-21swapper/009:39:110
1865899155,9cyclictest1142-21runrttasks07:23:162
1865899154,10cyclictest9418-21cut07:55:212
1865899151,6cyclictest11215-21sed08:00:162
1865899151,13cyclictest27898-21grep08:35:152
1865599159,1cyclictest25-21ksoftirqd/112:14:261
1865599157,5cyclictest13926-21apache211:17:201
18655991514,1cyclictest241rcuc/110:13:101
18655991511,4cyclictest0-21swapper/110:33:451
1865599150,2cyclictest0-21swapper/109:40:141
1865399153,7cyclictest4256-21perf08:55:000
18653991515,0cyclictest0-21swapper/012:32:090
1865399150,1cyclictest0-21swapper/010:15:140
1866399147,4cyclictest13926-21apache210:05:103
1866399143,9cyclictest22395-21apt08:25:013
18663991411,2cyclictest29109-21sed08:35:263
18663991410,3cyclictest8340-21grep07:55:153
18663991410,3cyclictest379-21grep10:20:173
18663991410,3cyclictest30533-21sed07:30:263
1865899149,4cyclictest16842-21cat08:10:232
1865899149,4cyclictest16842-21cat08:10:222
1865899148,5cyclictest30940-21grep08:40:202
1865899148,5cyclictest21624-21tr07:15:152
1865899147,6cyclictest26554-21cut07:25:152
1865899140,14cyclictest0-21swapper/208:30:202
1865899140,13cyclictest0-21swapper/207:30:122
18655991410,2cyclictest25-21ksoftirqd/111:38:251
1865599140,2cyclictest25-21ksoftirqd/112:31:591
1865599140,1cyclictest0-21swapper/111:10:191
1865599140,1cyclictest0-21swapper/110:37:381
1865399142,12cyclictest0-21swapper/012:38:480
18653991412,2cyclictest13-21ksoftirqd/010:06:070
18653991411,2cyclictest9396-21tail07:55:210
1865399140,3cyclictest0-21swapper/009:51:390
1865399140,13cyclictest0-21swapper/011:36:240
1865399140,12cyclictest0-21swapper/011:50:110
18663991310,2cyclictest7606-21head09:00:173
18663991310,2cyclictest2821-21tune2fs08:50:163
18663991310,2cyclictest13851-21grep08:05:183
18663991310,2cyclictest1327-21grep10:40:183
1865899139,3cyclictest23886-21grep08:25:222
1865599139,3cyclictest23104-21cat09:55:161
1865599139,2cyclictest1851-21cron08:50:001
1865599131,7cyclictest27146-21awk08:35:011
18655991310,2cyclictest26968-21tr10:55:191
18655991310,2cyclictest25-21ksoftirqd/108:15:011
18655991310,2cyclictest25-21ksoftirqd/108:15:011
1865399139,3cyclictest22546-21grep12:10:150
1865399139,2cyclictest23132-21tail07:15:270
1865399138,2cyclictest1300-21sh09:41:390
18653991310,2cyclictest6198-21ssh11:40:320
18653991310,2cyclictest28580-21basename07:30:000
18653991310,2cyclictest2476-21taskset10:01:530
18653991310,2cyclictest2476-21taskset10:01:520
18653991310,2cyclictest20100-21ls07:10:350
18653991310,2cyclictest17997-21wc11:10:000
1865399130,2cyclictest0-21swapper/010:51:130
1865399130,13cyclictest0-21swapper/012:01:510
1866399129,2cyclictest31450-21grep07:35:163
1866399129,2cyclictest30349-21df08:40:153
1866399129,2cyclictest26718-21grep07:25:153
1866399129,2cyclictest26682-21grep08:30:263
1866399129,2cyclictest1472-21grep08:45:273
1866399129,2cyclictest14178-21sort12:05:153
1866399126,1cyclictest31139-21apt-get12:35:153
1866399125,2cyclictest225-21systemd-journal09:09:083
1865899129,2cyclictest2849-21grep07:40:262
1865599129,2cyclictest9973-21fschecks_time09:05:181
1865599129,2cyclictest7615-21grep07:50:261
1865599129,2cyclictest2164-21cut08:50:141
1865599129,2cyclictest19274-21grep10:50:261
1865599128,3cyclictest13858-21grep08:05:181
1865599125,7cyclictest0-21swapper/111:32:471
1865599124,7cyclictest8560-21sed09:00:261
1865599122,6cyclictest25-21ksoftirqd/108:55:191
1865599122,6cyclictest25-21ksoftirqd/107:55:211
1865599121,3cyclictest25-21ksoftirqd/107:33:081
18655991211,1cyclictest38150irq/131-snd_hda08:40:141
18655991211,1cyclictest241rcuc/111:05:501
1865599121,10cyclictest31355-21cut07:35:151
1865599120,1cyclictest0-21swapper/107:24:131
1865599120,12cyclictest16413-21awk12:25:271
1865399129,2cyclictest30545-21dump-pmu-power10:00:000
1865399129,2cyclictest29514-21grep07:30:170
1865399129,2cyclictest24301-21sed08:25:260
1865399129,2cyclictest20005-21dump-pmu-power08:20:000
1865399129,2cyclictest15859-21taskset08:10:150
1865399129,2cyclictest15859-21taskset08:10:150
1865399129,1cyclictest30215-21apache_processe12:15:140
1865399128,2cyclictest6653-21grep09:25:270
1865399124,6cyclictest11388-21diskmemload11:15:160
1865399120,11cyclictest27152-21ls08:35:010
1866399118,2cyclictest9100-21ssh12:21:153
1866399118,2cyclictest7642-21head07:50:273
1866399118,2cyclictest3828-21tr07:45:163
1866399118,2cyclictest27890-21ssh11:35:033
1866399118,2cyclictest27047-21ssh10:55:203
1866399118,2cyclictest25198-21head07:20:203
1866399118,2cyclictest2505-21fgrep07:40:223
1866399118,2cyclictest23439-21cut08:25:183
1866399118,2cyclictest20790-21ssh11:50:133
1866399118,2cyclictest19993-21cron08:20:003
1866399118,2cyclictest19150-21taskset09:33:273
1866399118,2cyclictest15998-21ssh10:30:013
1866399118,2cyclictest15847-21ssh11:27:273
1866399110,11cyclictest0-21swapper/310:46:293
1865599118,2cyclictest25273-21sensors10:15:231
1865599118,2cyclictest25-21ksoftirqd/108:40:011
1865599118,2cyclictest23392-21grep08:25:171
1865599118,2cyclictest21964-21df07:15:161
1865599118,2cyclictest1974-21head07:40:171
1865599118,2cyclictest19135-21grep07:10:291
1865599115,3cyclictest19868-21taskset08:18:261
1865599114,6cyclictest11388-21diskmemload11:25:031
1865399119,1cyclictest2122-21uname07:40:180
1865399118,2cyclictest9787-21chrt12:21:490
1865399118,2cyclictest6602-21cron09:00:000
1865399118,2cyclictest5836-21sh09:45:020
1865399118,2cyclictest5768-21ssh11:21:090
1865399118,2cyclictest5415-21rm11:01:500
1865399118,2cyclictest4108-21df07:45:170
1865399118,2cyclictest25629-21chrt10:55:090
1865399118,2cyclictest25616-21ssh10:35:180
1865399118,2cyclictest2023-21ssh11:57:590
1865399118,2cyclictest16582-21ssh11:28:030
1865399118,2cyclictest15223-21cat12:05:250
1865399118,2cyclictest14501-21cat09:30:210
1865399118,2cyclictest14171-21rm12:25:020
1865399118,2cyclictest13678-21grep11:45:200
1865399113,4cyclictest8310-21apt-get10:45:020
1865399113,4cyclictest225-21systemd-journal07:20:000
1865399110,10cyclictest18956-21ssh10:30:580
1866399108,1cyclictest20781-21taskset07:12:233
1866399108,1cyclictest15481-21basename08:10:123
1866399108,1cyclictest15481-21basename08:10:113
1866399107,2cyclictest23178-21grep07:15:273
1865599107,2cyclictest5586-21seq07:49:091
1865599106,1cyclictest22171-21taskset08:22:141
1865599105,5cyclictest25-21ksoftirqd/108:05:011
1865399108,2cyclictest13-21ksoftirqd/007:35:230
1865399108,1cyclictest8095-21cut09:00:210
1865399108,1cyclictest31626-21taskset08:41:280
1865399108,1cyclictest13973-21if_enp0s31f608:05:180
1865399107,2cyclictest6248-21cat07:50:140
1865399107,2cyclictest32574-21cat08:45:160
1865399107,2cyclictest29273-21taskset08:37:030
1865399107,2cyclictest15157-21taskset10:10:040
1865399107,2cyclictest12984-21ssh09:10:190
1865399107,2cyclictest11393-21grep09:09:590
1865399100,7cyclictest22399-21sh08:25:010
186639997,1cyclictest5143-21grep08:55:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional