You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 20:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun Aug 30, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
119129481,9sleep00-21swapper/007:09:060
100028975,9sleep20-21swapper/207:06:442
93428469,10sleep30-21swapper/307:05:533
117728265,13sleep10-21swapper/107:08:551
135699250,22cyclictest0-21swapper/111:29:261
135699210,19cyclictest0-21swapper/110:10:111
135399210,21cyclictest0-21swapper/010:04:180
135399210,1cyclictest0-21swapper/011:27:340
1358992018,2cyclictest0-21swapper/211:46:332
135899200,19cyclictest0-21swapper/210:55:422
135699200,1cyclictest0-21swapper/109:31:221
1353992019,1cyclictest0-21swapper/010:42:510
135399200,1cyclictest0-21swapper/011:50:150
135899194,8cyclictest22918-21df_abs10:05:152
135899190,1cyclictest0-21swapper/211:09:162
1356991915,2cyclictest16784-21sh12:35:291
296362180,0sleep00-21swapper/009:10:500
1362991816,1cyclictest16701-21kworker/3:110:05:233
1358991813,1cyclictest32-21ksoftirqd/209:35:442
135899180,18cyclictest0-21swapper/211:27:272
135699180,1cyclictest0-21swapper/110:50:481
135399180,1cyclictest0-21swapper/012:04:410
135399180,1cyclictest0-21swapper/011:09:150
1362991716,1cyclictest0-21swapper/308:50:223
136299170,16cyclictest0-21swapper/307:25:143
135899170,17cyclictest0-21swapper/211:38:572
135699170,2cyclictest0-21swapper/111:35:501
135399170,16cyclictest0-21swapper/012:37:460
1358991615,1cyclictest0-21swapper/212:05:222
1358991615,1cyclictest0-21swapper/209:27:502
1358991611,3cyclictest0-21swapper/209:22:062
135699169,6cyclictest24421-21grep10:25:181
135699166,4cyclictest26579-21diskmemload12:02:421
135699160,16cyclictest0-21swapper/111:08:151
135699160,14cyclictest0-21swapper/110:35:141
135399164,6cyclictest16097-21awk07:40:010
135399164,6cyclictest12378-21munin-run09:40:010
136299152,12cyclictest0-21swapper/309:50:303
1362991514,1cyclictest16701-21kworker/3:111:14:243
136299151,1cyclictest2123-21ssh10:30:553
136299151,1cyclictest0-21swapper/309:26:033
1358991512,2cyclictest31323-21df10:10:142
135899151,1cyclictest141rcu_preempt11:00:262
135699153,11cyclictest26579-21diskmemload09:15:151
1356991512,2cyclictest3354-21cat11:10:261
135699151,13cyclictest26579-21diskmemload12:26:231
135699150,14cyclictest0-21swapper/109:43:001
1353991514,1cyclictest0-21swapper/010:12:370
135399150,15cyclictest0-21swapper/011:30:070
136299149,3cyclictest12827-21wc07:30:263
136299145,6cyclictest1235-21ntpd10:20:143
136299143,6cyclictest30810-21ssh09:30:353
1362991413,1cyclictest39-21ksoftirqd/311:17:133
1362991410,3cyclictest17358-21ssh11:00:153
136299140,14cyclictest0-21swapper/310:29:233
1358991412,1cyclictest27128-21apt-get09:10:152
1358991411,3cyclictest32-21ksoftirqd/207:35:122
1358991411,2cyclictest16027-21head10:20:162
1358991410,3cyclictest30395-21wc08:10:002
1358991410,3cyclictest18534-21grep11:20:132
135699149,3cyclictest27419-21tr09:10:161
1356991411,2cyclictest7442-21grep09:55:281
135699140,14cyclictest0-21swapper/110:30:421
135699140,13cyclictest0-21swapper/109:29:231
135399141,7cyclictest18480-21munin-run07:45:000
1353991412,1cyclictest0-21swapper/010:59:070
136299139,3cyclictest2279-21sed08:15:253
136299139,3cyclictest17113-21sed07:40:183
136299137,5cyclictest26300-21apt-config11:25:003
136299137,3cyclictest0-21swapper/312:30:143
136299132,5cyclictest11300-21ssh09:20:003
1362991312,1cyclictest16701-21kworker/3:111:06:533
1362991310,2cyclictest6389-21grep09:55:183
1362991310,2cyclictest29322-21basename12:05:133
1362991310,2cyclictest2547-21cron09:15:003
1362991310,2cyclictest23339-21ls12:20:223
1362991310,2cyclictest22116-21head09:45:183
1362991310,2cyclictest22116-21head09:45:173
136299130,12cyclictest5787-21idleruntime-cro12:30:003
135899138,1cyclictest32-21ksoftirqd/212:15:162
1358991310,2cyclictest9358-21wc10:35:232
1358991310,2cyclictest31645-21tr12:25:212
1358991310,2cyclictest30597-21grep09:30:272
1358991310,2cyclictest30050-21rm09:50:132
1358991310,2cyclictest26556-21cut08:00:172
1358991310,2cyclictest21882-21cat08:59:592
1358991310,2cyclictest17767-21cron11:20:012
135699139,3cyclictest27577-21sed08:00:271
135699139,3cyclictest14251-21wc12:35:001
135699139,3cyclictest10920-21ssh11:15:201
1356991313,0cyclictest0-21swapper/111:42:131
1356991313,0cyclictest0-21swapper/110:56:481
1356991311,2cyclictest25-21ksoftirqd/111:45:591
1356991310,2cyclictest4802-21ssh12:10:001
1356991310,2cyclictest23892-21grep12:20:261
1356991310,2cyclictest14817-21grep10:00:161
1356991310,2cyclictest10648-21head07:25:271
135399139,2cyclictest2904-21sed11:10:210
135399136,6cyclictest30210-21mailstats09:30:230
135399135,7cyclictest32763-21perf08:15:000
135399133,4cyclictest0-21swapper/009:05:150
135399131,6cyclictest5110-21munin-run08:25:000
1353991310,2cyclictest26293-21ls11:25:000
1353991310,2cyclictest25099-21sed07:55:270
1353991310,2cyclictest25099-21sed07:55:260
1353991310,2cyclictest2363-21sed07:10:180
1353991310,2cyclictest20573-21wc09:25:150
1353991310,2cyclictest18161-21wc11:00:180
1353991310,2cyclictest12183-21cut11:35:170
1353991310,2cyclictest11367-21wc08:35:210
135399130,12cyclictest32198-21tr10:30:150
136299129,2cyclictest22415-21cut12:00:233
136299129,2cyclictest1613-21basename07:10:143
136299129,2cyclictest13694-21tr08:40:213
136299129,2cyclictest12779-21head09:20:203
136299122,6cyclictest7268-21df_abs10:15:153
136299120,12cyclictest0-21swapper/309:42:313
136299120,11cyclictest3609-21cut08:20:153
136299120,11cyclictest25548-21grep09:05:203
135899129,2cyclictest7326-21sed10:15:152
135899129,2cyclictest3502-21grep11:30:172
135899129,2cyclictest26558-21cron09:09:592
135899129,2cyclictest25373-21tail10:45:162
135899129,2cyclictest24341-21wc09:05:012
135899129,2cyclictest23706-21sh11:42:042
135899129,2cyclictest23542-21sensors12:20:232
135899129,2cyclictest19211-21seq10:02:562
135899129,2cyclictest13574-21wc11:55:182
135899129,2cyclictest13323-21head08:40:172
135899128,3cyclictest31394-21cron10:30:002
135899128,2cyclictest31345-21grep08:10:172
135899123,9cyclictest311rcuc/208:20:132
1358991210,1cyclictest2370-21sed07:10:172
135899120,12cyclictest0-21swapper/207:55:152
135899120,12cyclictest0-21swapper/207:55:142
135699129,2cyclictest26048-21rt-features10:45:211
135699129,2cyclictest24291-21grep10:05:261
135699129,2cyclictest2313-21tail07:10:171
135699129,2cyclictest19925-21cat09:25:001
135699129,2cyclictest18260-21grep10:40:261
135699129,2cyclictest14212-21grep07:35:141
135699129,2cyclictest12828-21ssh11:55:141
135699129,2cyclictest12059-21head07:30:181
135699129,2cyclictest11941-21sort11:55:011
135699128,4cyclictest38150irq/131-snd_hda09:00:151
135699128,3cyclictest30255-21ssh09:50:151
135699127,4cyclictest18803-21ssh10:21:411
135699121,10cyclictest12752-21apache209:39:071
1356991210,2cyclictest25-21ksoftirqd/109:49:341
1356991210,2cyclictest25-21ksoftirqd/109:49:331
1356991210,1cyclictest14669-21sh08:45:001
135699120,1cyclictest0-21swapper/111:24:151
135399129,2cyclictest7491-21tail09:55:280
135399129,2cyclictest6136-21tail08:25:170
135399129,2cyclictest31185-21grep12:25:180
135399129,2cyclictest25523-21sed10:25:270
135399129,2cyclictest2201-21grep08:15:250
135399129,2cyclictest20911-21ntpq11:40:210
135399129,2cyclictest16475-21cat08:45:240
135399129,2cyclictest16441-21cut10:20:190
135399129,2cyclictest14148-21sed12:15:160
135399129,2cyclictest12691-21grep09:20:190
135399128,3cyclictest20840-21cron07:50:010
135399128,2cyclictest22182-21grep09:00:130
135399128,2cyclictest14954-21unixbench_singl09:40:270
135399120,11cyclictest21854-21sed09:45:160
135399120,11cyclictest21854-21sed09:45:150
136299118,2cyclictest86512sleep307:24:473
136299118,2cyclictest8235-21ssh12:11:173
136299118,2cyclictest8139-21tr10:35:153
136299118,2cyclictest5115-21grep09:35:153
136299118,2cyclictest4692-21grep11:50:173
136299118,2cyclictest32724-21ssh10:50:043
136299118,2cyclictest30617-21ssh11:45:583
136299118,2cyclictest30458-21rm10:10:003
136299118,2cyclictest29745-21ssh11:26:093
136299118,2cyclictest26980-21grep08:00:203
136299118,2cyclictest26063-21wc10:45:223
136299118,2cyclictest23858-21chrt09:00:573
136299118,2cyclictest23770-21tr07:55:153
136299118,2cyclictest23770-21tr07:55:153
136299118,2cyclictest22493-21ssh11:41:073
136299118,2cyclictest21342-21grep07:50:133
136299118,2cyclictest20841-21sh07:50:003
136299118,2cyclictest19723-21ssh12:37:463
136299118,2cyclictest19184-21ssh10:41:093
136299118,2cyclictest16267-21ssh12:15:573
136299118,2cyclictest14901-21ssh11:36:453
136299118,2cyclictest12486-21ssh10:56:433
136299118,2cyclictest12215-21ssh11:55:033
136299118,2cyclictest11939-21chrt08:36:363
136299118,2cyclictest10255-21ssh11:34:423
136299116,4cyclictest0-21swapper/308:55:143
1362991110,1cyclictest0-21swapper/310:00:143
135899118,2cyclictest8483-21ssh09:17:572
135899118,2cyclictest7226-21taskset08:27:032
135899118,2cyclictest5547-21ssh09:55:142
135899118,2cyclictest5038-21ssh11:11:382
135899118,2cyclictest3997-21ssh10:32:232
135899118,2cyclictest32691-21ssh10:50:042
135899118,2cyclictest23603-21ssh09:45:412
135899118,2cyclictest23603-21ssh09:45:402
135899118,2cyclictest23005-21ssh12:00:332
135899118,2cyclictest22754-21cat12:40:012
135899118,2cyclictest17606-21ssh09:42:312
135899118,2cyclictest16911-21df07:40:172
135899118,2cyclictest16609-21grep10:40:152
135899118,2cyclictest16477-21wc08:45:242
135899118,2cyclictest11432-21sh12:32:522
135899118,2cyclictest10551-21ls08:35:162
135899118,2cyclictest10179-21ssh11:53:502
135899117,3cyclictest5848-21cut12:10:172
135899115,4cyclictest18668-21/usr/sbin/munin08:50:232
135899113,4cyclictest225-21systemd-journal07:55:002
135699119,1cyclictest7560-21head10:15:171
135699118,2cyclictest7997-21tr08:30:141
135699118,2cyclictest7294-21head07:20:171
135699118,2cyclictest24135-21cut07:55:181
135699118,2cyclictest24135-21cut07:55:171
135699118,2cyclictest23304-21ssh11:03:371
135699118,2cyclictest23224-21apt07:55:001
135699118,2cyclictest20809-21tr08:55:191
135699118,2cyclictest19319-21ssh12:18:201
135699118,2cyclictest15334-21cat08:45:131
135699118,2cyclictest10720-21wc11:35:001
135699117,3cyclictest12305-21cat08:39:591
135699117,2cyclictest16650-21grep07:40:151
135699110,0cyclictest26579-21diskmemload12:12:211
135399118,2cyclictest8749-21wc07:25:010
135399118,2cyclictest7769-21taskset10:35:090
135399118,2cyclictest574-21cut10:50:140
135399118,2cyclictest4898-21uname07:15:180
135399118,2cyclictest31994-21ssh11:47:020
135399118,2cyclictest31539-21grep09:50:250
135399118,2cyclictest2950-21sort09:15:010
135399118,2cyclictest29068-21sh12:05:060
135399118,2cyclictest28774-21ssh12:24:100
135399118,2cyclictest28756-21ssh10:08:520
135399118,2cyclictest26523-21sed10:45:250
135399118,2cyclictest21503-21cat07:50:130
135399118,2cyclictest18994-21grep08:50:270
135399118,2cyclictest14374-21taskset11:17:220
135399118,2cyclictest13367-21ssh10:19:020
135399118,2cyclictest11969-21cut07:30:180
135399110,10cyclictest6334-21ssh12:10:190
136299108,1cyclictest7742-21dpkg08:30:133
136299108,1cyclictest305912sleep308:10:083
136299108,1cyclictest29259-21cat08:05:193
136299108,1cyclictest14823-21if_enp0s31f607:35:173
136299107,2cyclictest5783-21expr08:25:163
136299107,2cyclictest4187-21cut07:15:143
136299107,2cyclictest15161-21grep08:45:143
135899108,1cyclictest7055-21df_abs07:20:142
135899108,1cyclictest1625-21expr08:15:192
135899108,1cyclictest10697-21chrt07:25:342
135899107,2cyclictest8503-21head08:30:172
135899107,2cyclictest4235-21cut07:15:142
135899107,2cyclictest19366-21grep07:45:192
135899107,2cyclictest12095-21expr07:30:182
135699108,1cyclictest854-21grep08:15:141
135699108,1cyclictest29557-21uname08:05:221
135699108,1cyclictest20083-21tr07:45:251
135699107,2cyclictest4692-21expr07:15:161
135699107,2cyclictest32632-21taskset08:13:481
135699107,2cyclictest26586-21awk09:09:591
135699107,2cyclictest18468-21ntpq08:50:211
135699104,5cyclictest3640-21fschecks_time08:20:151
135399108,1cyclictest6228-21ssh12:30:090
135399108,1cyclictest27914-21chrt08:04:110
135399107,2cyclictest9732-21chrt08:33:450
135399107,2cyclictest28953-21cut08:05:170
135399107,2cyclictest12579-21ssh11:55:140
135399103,6cyclictest0-21swapper/008:59:590
135399100,9cyclictest10874-21chrt07:27:530
13569986,1cyclictest5779-21expr08:25:161
13539986,1cyclictest145662sleep008:43:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional