You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 09:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Mon May 25, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
779929078,8sleep00-21swapper/019:05:070
976728975,9sleep20-21swapper/219:07:542
977128867,6sleep10-21swapper/119:07:571
960728661,9sleep30-21swapper/319:05:503
272182600,0sleep20-21swapper/200:37:562
1001899320,0cyclictest0-21swapper/119:12:251
1002099280,28cyclictest0-21swapper/222:21:382
1002099270,27cyclictest0-21swapper/221:58:162
1002099270,0cyclictest0-21swapper/223:20:362
1002099260,26cyclictest0-21swapper/222:33:152
1002099260,24cyclictest0-21swapper/222:51:232
1002099251,23cyclictest3031-21grep23:45:142
1002099250,1cyclictest0-21swapper/221:20:012
1002099240,24cyclictest0-21swapper/222:17:122
1002099232,21cyclictest0-21swapper/223:11:532
1002099230,23cyclictest0-21swapper/223:06:232
1002099230,23cyclictest0-21swapper/200:18:342
1002399220,22cyclictest0-21swapper/322:35:383
1002099220,22cyclictest0-21swapper/200:10:212
1002099220,21cyclictest0-21swapper/222:59:232
1002099220,21cyclictest0-21swapper/221:21:232
1001399220,3cyclictest0-21swapper/000:27:390
1001899210,20cyclictest0-21swapper/122:15:181
1002399204,12cyclictest39-21ksoftirqd/322:13:013
1002099200,20cyclictest0-21swapper/223:04:262
1002099200,0cyclictest0-21swapper/221:42:162
1002099200,0cyclictest0-21swapper/221:33:312
1002099190,1cyclictest0-21swapper/200:29:202
1002099190,19cyclictest0-21swapper/200:08:162
1002099190,15cyclictest0-21swapper/223:43:262
1002099190,0cyclictest0-21swapper/222:00:332
1001899190,18cyclictest0-21swapper/121:50:181
1001899190,18cyclictest0-21swapper/121:45:121
1002099180,18cyclictest0-21swapper/223:15:032
1002099180,18cyclictest0-21swapper/222:12:402
1001899180,17cyclictest0-21swapper/122:33:561
1001899180,15cyclictest0-21swapper/122:59:131
10013991816,1cyclictest537-21snmpd23:46:110
1001399180,2cyclictest0-21swapper/000:07:250
1002399172,2cyclictest141rcu_preempt21:29:013
10023991716,1cyclictest0-21swapper/321:47:453
10023991715,1cyclictest0-21swapper/321:15:273
10023991711,3cyclictest0-21swapper/322:30:193
1002399170,1cyclictest0-21swapper/300:00:533
1002099170,1cyclictest0-21swapper/222:27:342
1002099170,16cyclictest0-21swapper/200:04:422
1001899172,14cyclictest0-21swapper/121:32:341
10018991711,6cyclictest10005-21cyclictest22:44:061
1002399164,8cyclictest25225-21cron23:20:013
10020991612,3cyclictest55750irq/128-enp2s0-23:54:452
1002099160,3cyclictest0-21swapper/221:37:482
1002099160,16cyclictest0-21swapper/223:25:332
1002099160,0cyclictest0-21swapper/222:40:352
1002099160,0cyclictest0-21swapper/221:53:532
1001399160,2cyclictest0-21swapper/023:10:590
1001399160,1cyclictest0-21swapper/021:57:160
1002399151,5cyclictest225-21systemd-journal23:03:183
1002399150,2cyclictest0-21swapper/323:11:363
10020991514,1cyclictest0-21swapper/220:05:142
10020991512,2cyclictest31050-21ls19:50:262
1002099150,13cyclictest0-21swapper/223:34:182
1001899154,9cyclictest537-21snmpd21:28:161
1001899151,14cyclictest14074-21sh23:31:531
1001899150,15cyclictest0-21swapper/122:00:191
1001399154,6cyclictest13781-21perf20:24:590
10023991411,2cyclictest18318-21grep19:25:173
1002399140,13cyclictest0-21swapper/321:40:133
1002099149,4cyclictest7648-21wc20:10:192
10020991410,3cyclictest31888-21basename19:55:152
10020991410,3cyclictest29771-21wc20:55:242
10020991410,2cyclictest30519-21wc21:45:182
1002099140,1cyclictest0-21swapper/219:34:452
1002099140,13cyclictest0-21swapper/200:20:562
10018991412,2cyclictest25-21ksoftirqd/122:21:051
10018991412,1cyclictest38150irq/131-snd_hda23:10:151
1001899141,12cyclictest2759-21diskmemload00:06:261
10013991411,2cyclictest19240-21rm23:15:250
1001399140,7cyclictest12375-21awk19:15:000
1001399140,1cyclictest0-21swapper/000:37:140
1001399140,11cyclictest26799-21apache_processe23:40:120
56722130,2sleep1241rcuc/122:09:591
1002399134,5cyclictest225-21systemd-journal22:19:593
1002399133,4cyclictest0-21swapper/323:00:003
1002399131,4cyclictest20343-21fschecks_count23:55:163
10023991312,1cyclictest0-21swapper/321:59:423
10023991310,2cyclictest3976-21awk20:04:593
10023991310,2cyclictest25016-21sh22:20:343
10023991310,2cyclictest22085-21cron19:35:013
10023991310,2cyclictest20972-21grep20:40:013
10023991310,2cyclictest20499-21grep23:35:283
10023991310,2cyclictest15686-21grep00:30:263
1002399130,1cyclictest0-21swapper/320:40:183
1002399130,13cyclictest0-21swapper/323:43:523
1002099139,3cyclictest30516-21tr22:05:152
1002099138,3cyclictest28610-21wc19:45:242
10020991310,2cyclictest25643-21cron20:49:592
10020991310,2cyclictest23471-21uname19:35:212
10020991310,2cyclictest21138-21cut23:55:222
1002099130,2cyclictest0-21swapper/222:45:402
1002099130,13cyclictest0-21swapper/200:33:262
1001899139,4cyclictest38150irq/131-snd_hda23:57:401
1001899134,8cyclictest0-21swapper/123:18:041
1001899132,1cyclictest22659-21ssh21:20:591
10018991312,1cyclictest0-21swapper/121:41:541
10018991311,2cyclictest38150irq/131-snd_hda00:36:181
10018991311,1cyclictest0-21swapper/123:00:271
1001899130,11cyclictest0-21swapper/121:56:531
1001399139,3cyclictest25795-21grep22:40:260
1001399139,3cyclictest18262-21uniq19:25:160
1001399132,6cyclictest9761-21munin-run23:30:000
1001399131,9cyclictest6263-21wget22:10:140
10013991312,1cyclictest0-21swapper/021:46:270
10013991310,2cyclictest6391-21grep21:30:260
10013991310,2cyclictest23385-21wc20:45:010
10013991310,2cyclictest15378-21tail22:15:160
1001399130,9cyclictest0-21swapper/021:16:340
1001399130,3cyclictest7234-21wget22:30:130
1001399130,1cyclictest0-21swapper/022:24:580
1001399130,1cyclictest0-21swapper/021:24:350
1001399130,11cyclictest0-21swapper/023:30:540
1001399130,10cyclictest29028-21/usr/sbin/munin00:00:190
185052120,1sleep20-21swapper/222:36:142
1002399129,2cyclictest7587-21cron22:50:013
1002399129,2cyclictest6410-21wc21:50:193
1002399129,2cyclictest5227-21tail21:30:173
1002399129,2cyclictest14129-21grep20:25:133
1002399129,2cyclictest13566-21tail21:35:173
1002399129,2cyclictest13173-21df19:15:153
1002399129,2cyclictest11111-21grep23:30:193
1002399123,6cyclictest22236-21wget00:35:173
1002399121,2cyclictest0-21swapper/322:50:163
1002399120,11cyclictest29759-21tr00:20:143
1002099129,2cyclictest5243-21sed21:10:282
1002099129,2cyclictest2878-21wc20:00:182
1002099129,2cyclictest26418-21tail19:40:262
1002099128,2cyclictest19730-21awk19:30:002
1001899129,2cyclictest6475-21tr22:10:151
1001899129,2cyclictest32299-21grep21:00:261
1001899129,2cyclictest30416-21readlink21:00:001
1001899129,2cyclictest30228-21grep19:50:171
1001899129,2cyclictest28026-21sh00:00:131
1001899129,2cyclictest27221-21gzip23:20:221
1001899129,2cyclictest24029-21df20:45:151
1001899129,2cyclictest1573-21awk20:00:011
1001899129,2cyclictest15474-21wc00:30:261
1001899120,12cyclictest3485-21ssh00:23:571
1001899120,11cyclictest0-21swapper/123:05:011
1001399129,2cyclictest663-21ls19:55:230
1001399129,2cyclictest29909-21grep20:55:260
1001399129,2cyclictest29384-21cat19:50:010
1001399129,2cyclictest23077-21grep19:35:170
1001399129,2cyclictest19341-21taskset22:36:520
1001399129,2cyclictest17125-21head20:30:160
1001399129,2cyclictest14178-21tr21:35:220
1001399129,2cyclictest10259-21ls20:15:210
1001399127,2cyclictest452-21systemd-logind00:10:590
1001399122,9cyclictest15467-21apache200:22:310
10013991212,0cyclictest0-21swapper/023:53:080
1001399120,11cyclictest6514-21ntpq21:50:200
1001399120,11cyclictest21856-21perl21:40:150
1002399118,2cyclictest9754-21tail20:15:173
1002399118,2cyclictest8357-21head20:10:263
1002399118,2cyclictest7328-21chrt00:25:273
1002399118,2cyclictest702-21wc19:55:233
1002399118,2cyclictest6030-21taskset20:05:283
1002399118,2cyclictest4275-21ssh23:06:423
1002399118,2cyclictest3998-21ssh23:45:203
1002399118,2cyclictest32504-21ssh22:05:413
1002399118,2cyclictest31996-21grep22:25:163
1002399118,2cyclictest28184-21wc19:45:183
1002399118,2cyclictest26052-21sh23:20:133
1002399118,2cyclictest26040-21ssh22:02:073
1002399118,2cyclictest25489-21grep19:40:173
1002399118,2cyclictest24555-21cut22:40:153
1002399118,2cyclictest23777-21tr20:45:153
1002399118,2cyclictest2252-21grep21:05:263
1002399118,2cyclictest21943-21awk21:20:253
1002399118,2cyclictest20592-21ls00:15:003
1002399118,2cyclictest18525-21basename20:34:593
1002399118,2cyclictest17421-21sh23:53:393
1002399118,2cyclictest16295-21wc19:20:183
1002399118,2cyclictest10891-21cut19:10:163
1002399117,3cyclictest3532-21wc23:25:253
1002399116,4cyclictest15454-21apache200:17:463
1002099119,1cyclictest20316-21grep20:35:272
1002099118,2cyclictest9796-21sed20:15:172
1002099118,2cyclictest31283-21sort21:00:172
1002099118,2cyclictest22115-21ssh23:36:382
1002099118,2cyclictest17340-21cron19:25:002
1002099118,2cyclictest15343-21sendmail_mailqu20:25:222
1002099118,2cyclictest13848-21grep19:15:222
1002099118,2cyclictest13299-21sed20:20:262
1002099117,3cyclictest18510-21apt20:34:592
1002099117,3cyclictest11236-21wc19:10:192
1002099110,4cyclictest27805-21sshd20:52:202
1002099110,2cyclictest0-21swapper/221:27:242
1001899119,1cyclictest634-21grep22:25:261
1001899119,1cyclictest22223-21cat00:15:211
1001899119,1cyclictest16663-21grep20:30:141
1001899118,2cyclictest9979-21ssh23:49:321
1001899118,2cyclictest29403-21ssh23:40:531
1001899118,2cyclictest22128-21cat20:40:201
1001899118,2cyclictest18735-21taskset20:35:071
1001899118,2cyclictest15916-21cron22:55:011
1001899118,2cyclictest13722-21seq20:24:411
1001899118,2cyclictest12799-21aten2_r7power_c21:35:141
1001899118,2cyclictest12094-21grep23:50:151
1001899118,2cyclictest11389-21ssh00:28:381
1001899117,3cyclictest8678-21ssh23:29:101
1001899116,4cyclictest22790-21expr19:35:151
1001899115,3cyclictest4835-21apache200:10:121
1001899110,2cyclictest0-21swapper/122:36:131
1001399118,2cyclictest5928-21ssh23:07:570
1001399118,2cyclictest4233-21ssh22:28:090
1001399118,2cyclictest30590-21ssh23:22:410
1001399118,2cyclictest28634-21ssh00:19:490
1001399118,2cyclictest26868-21apt19:45:000
1001399118,2cyclictest23985-21ssh22:00:300
1001399118,2cyclictest23855-21expr20:45:140
1001399118,2cyclictest20584-21ssh00:34:160
1001399118,2cyclictest19992-21cut23:55:130
1001399118,2cyclictest1624-21tail22:45:260
1001399118,2cyclictest14788-21sed20:25:170
1001399118,2cyclictest14715-21taskset19:19:530
1001399118,2cyclictest11162-21ssh22:51:160
1001399115,3cyclictest4446-21memory21:10:210
1001399115,2cyclictest31302-21kernelversion22:05:200
1001399113,4cyclictest225-21systemd-journal19:25:010
1001399110,10cyclictest30313-21ssh23:03:120
1001399110,10cyclictest29674-21pmu-power21:25:210
1002399108,1cyclictest5589-21tr00:05:253
1002399108,1cyclictest29449-21chrt20:55:203
1002399107,2cyclictest3420-21cut21:10:163
1002399107,2cyclictest32382-21grep21:00:283
1002399107,2cyclictest30952-21sort19:50:263
1002399107,2cyclictest26735-21basename20:50:193
1002399107,2cyclictest24106-21chrt19:35:303
1002099108,1cyclictest1291-21uniq21:05:182
1002099103,3cyclictest225-21systemd-journal20:40:002
1001899108,2cyclictest32474-21expr22:45:151
1001899108,2cyclictest19267-21rm23:35:181
1001899108,2cyclictest14505-21sh20:25:151
1001899108,1cyclictest8947-21cat20:15:001
1001899108,1cyclictest2270-21expr20:00:131
1001899108,1cyclictest20322-21cat19:30:151
1001899108,1cyclictest16012-21sed19:20:161
1001899108,1cyclictest11569-21ssh21:15:101
1001899107,3cyclictest5390-21ssh21:10:291
1001899106,4cyclictest29092-21taskset19:48:181
1001899100,9cyclictest2704-21taskset21:09:351
1001899100,6cyclictest11086-21sshd20:19:191
1001399108,1cyclictest3793-21chrt20:03:010
1001399108,1cyclictest31294-21taskset19:51:050
1001399108,1cyclictest20817-21cat19:30:180
1001399107,2cyclictest5759-21threads20:05:260
1001399107,2cyclictest31020-21expr21:00:140
1001399107,2cyclictest26017-21wget20:50:150
1001399107,2cyclictest2304-21grep21:05:270
1001399107,2cyclictest18728-21seq20:35:020
1001399107,2cyclictest18469-21ssh23:35:090
1001399107,2cyclictest17882-21gzip22:55:210
1001399100,9cyclictest7037-21expr20:10:140
100239997,1cyclictest12079-21expr20:20:143
100189998,1cyclictest4630-21expr20:05:151
100189998,1cyclictest25880-21taskset20:50:091
100189998,1cyclictest18947-21systemctl19:25:221
100189997,2cyclictest25745-21cat19:40:191
100189990,9cyclictest14377-21seq19:15:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional