You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 21:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Tue May 26, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1839210877,26sleep00-21swapper/007:09:060
183029080,6sleep10-21swapper/107:08:591
174329077,9sleep20-21swapper/207:07:522
177128471,9sleep30-21swapper/307:08:133
181062750,0sleep30-21swapper/311:38:163
321452470,0sleep10-21swapper/109:50:311
200299360,35cyclictest21914-21kworker/3:111:06:333
2002993028,1cyclictest21914-21kworker/3:111:32:033
2002993027,2cyclictest0-21swapper/311:30:003
1996992922,5cyclictest25-21ksoftirqd/109:26:221
2002992827,1cyclictest4658-21kworker/3:009:59:033
200099280,28cyclictest0-21swapper/209:11:102
200099280,19cyclictest141rcu_preempt10:32:342
2002992724,2cyclictest4658-21kworker/3:009:42:433
199199270,0cyclictest0-21swapper/009:25:110
199199260,24cyclictest0-21swapper/011:59:540
2002992524,1cyclictest21914-21kworker/3:112:00:043
2002992523,1cyclictest21914-21kworker/3:111:14:233
2002992521,3cyclictest4658-21kworker/3:009:35:143
200299251,23cyclictest4658-21kworker/3:009:45:533
1996992419,1cyclictest25-21ksoftirqd/112:07:001
2002992319,3cyclictest21914-21kworker/3:110:06:033
200299231,22cyclictest21914-21kworker/3:110:48:333
200299230,23cyclictest4658-21kworker/3:009:19:133
200299220,20cyclictest0-21swapper/309:29:303
200099220,1cyclictest0-21swapper/212:19:502
200099220,19cyclictest0-21swapper/211:19:492
199199223,14cyclictest30888-21ssh12:05:140
2002992119,2cyclictest21914-21kworker/3:111:20:233
200299211,20cyclictest21914-21kworker/3:112:23:573
200299210,21cyclictest21914-21kworker/3:110:39:343
200299210,21cyclictest0-21swapper/310:18:203
1996992119,2cyclictest25-21ksoftirqd/110:10:201
199199210,21cyclictest0-21swapper/010:11:330
2002992019,1cyclictest4658-21kworker/3:010:01:143
2002992018,2cyclictest21914-21kworker/3:111:55:433
2002992018,2cyclictest21914-21kworker/3:111:04:043
200099200,3cyclictest0-21swapper/212:00:242
199699208,3cyclictest141rcu_preempt08:20:231
1996992018,2cyclictest25-21ksoftirqd/111:24:481
1996992017,1cyclictest141rcu_preempt09:31:511
1996992017,0cyclictest0-21swapper/112:39:531
199699200,1cyclictest0-21swapper/112:16:511
2002991917,2cyclictest21914-21kworker/3:112:05:233
2002991916,3cyclictest4658-21kworker/3:009:33:533
2000991917,1cyclictest0-21swapper/211:40:252
1996991917,2cyclictest25-21ksoftirqd/110:15:271
199699190,3cyclictest141rcu_preempt11:02:141
1991991919,0cyclictest0-21swapper/011:43:260
1991991919,0cyclictest0-21swapper/010:39:200
1991991914,2cyclictest0-21swapper/010:23:450
199199191,0cyclictest0-21swapper/012:33:290
199199190,3cyclictest0-21swapper/011:05:020
199199190,19cyclictest0-21swapper/009:54:280
2002991817,1cyclictest21914-21kworker/3:110:53:033
2002991816,2cyclictest21914-21kworker/3:111:45:333
2000991816,1cyclictest0-21swapper/212:31:002
200099180,17cyclictest0-21swapper/211:47:442
199699187,7cyclictest25-21ksoftirqd/109:40:341
199699187,3cyclictest25-21ksoftirqd/111:59:271
199699182,4cyclictest25-21ksoftirqd/112:26:401
1996991817,1cyclictest0-21swapper/112:22:001
1996991811,3cyclictest25-21ksoftirqd/111:08:091
1996991810,2cyclictest25-21ksoftirqd/109:16:391
199699180,15cyclictest0-21swapper/110:30:151
199199180,14cyclictest0-21swapper/011:50:140
199199180,13cyclictest0-21swapper/010:00:130
2002991716,1cyclictest21914-21kworker/3:111:53:343
2002991716,1cyclictest21914-21kworker/3:110:43:033
200099178,2cyclictest27221-21diskmemload10:05:132
200099173,1cyclictest27221-21diskmemload10:25:412
200099171,16cyclictest7904-21ssh11:12:442
199699174,3cyclictest141rcu_preempt10:23:221
199699173,7cyclictest25-21ksoftirqd/109:36:541
199699171,2cyclictest27221-21diskmemload12:32:011
199699170,17cyclictest0-21swapper/110:00:581
199199175,6cyclictest4932-21perf11:50:000
199199170,17cyclictest0-21swapper/009:59:560
200299164,8cyclictest7372-21df_abs12:10:173
200099166,8cyclictest5805-21runrttasks08:25:122
200099165,6cyclictest0-21swapper/210:55:132
2000991615,1cyclictest27221-21diskmemload11:22:342
2000991614,1cyclictest0-21swapper/209:17:442
200099160,16cyclictest0-21swapper/210:36:392
200099160,0cyclictest0-21swapper/212:11:592
1996991614,1cyclictest21307-21latency08:55:171
199699160,1cyclictest0-21swapper/111:38:081
199699160,15cyclictest0-21swapper/111:32:001
1991991615,1cyclictest0-21swapper/010:46:450
199199160,1cyclictest0-21swapper/011:19:200
200299155,3cyclictest225-21systemd-journal10:15:013
200299150,14cyclictest0-21swapper/312:36:263
200299150,14cyclictest0-21swapper/312:28:233
200099152,8cyclictest225-21systemd-journal11:35:002
199699157,2cyclictest0-21swapper/109:13:181
199699157,1cyclictest141rcu_preempt07:25:111
199699154,1cyclictest25-21ksoftirqd/111:16:001
199699152,2cyclictest141rcu_preempt12:00:221
1996991513,2cyclictest25-21ksoftirqd/107:30:131
1996991513,1cyclictest0-21swapper/109:48:301
199699151,1cyclictest141rcu_preempt10:05:011
199699151,1cyclictest141rcu_preempt07:23:481
1996991511,2cyclictest0-21swapper/109:24:131
199699151,11cyclictest27221-21diskmemload10:44:011
199699150,3cyclictest0-21swapper/111:29:291
199699150,1cyclictest0-21swapper/110:47:191
199699150,0cyclictest0-21swapper/112:11:241
1991991515,0cyclictest0-21swapper/010:06:530
1991991512,2cyclictest8005-21cut10:15:150
1991991511,3cyclictest15730-21df12:15:160
1991991510,3cyclictest28710-21tr08:05:170
199199150,1cyclictest0-21swapper/012:20:260
199199150,1cyclictest0-21swapper/011:24:430
153062150,1sleep0161rcuc/010:58:020
200299142,6cyclictest0-21swapper/312:30:463
2002991411,2cyclictest2332-21wget07:10:133
2002991410,3cyclictest3454-21grep08:20:123
2002991410,3cyclictest10189-21tr10:55:133
2000991413,1cyclictest0-21swapper/210:03:462
2000991410,2cyclictest19240-21grep07:45:162
200099140,1cyclictest0-21swapper/210:23:112
200099140,14cyclictest0-21swapper/211:54:022
199699148,2cyclictest25-21ksoftirqd/111:47:301
1996991411,2cyclictest5399-21head07:15:161
199699140,14cyclictest0-21swapper/108:10:161
199199149,4cyclictest0-21swapper/009:35:250
199199143,2cyclictest225-21systemd-journal12:39:270
199199141,7cyclictest14653-21perf12:15:000
199199140,13cyclictest10133-21grep08:30:250
200299139,3cyclictest17817-21grep07:40:203
200299139,3cyclictest11309-21grep08:35:153
2002991311,1cyclictest21914-21kworker/3:112:19:333
2002991310,2cyclictest8631-21grep08:30:133
2002991310,2cyclictest28895-21cat11:45:003
2002991310,2cyclictest28763-21grep08:05:173
2002991310,2cyclictest21091-21sed08:55:153
200099139,3cyclictest21742-21basename09:45:122
200099139,3cyclictest13888-21awk11:35:212
200099135,7cyclictest0-21swapper/212:08:012
2000991310,2cyclictest5715-21tail09:35:162
2000991310,2cyclictest22940-21grep07:50:252
200099130,12cyclictest0-21swapper/208:45:152
199699139,3cyclictest13431-21gpgv11:55:011
199699139,2cyclictest24352-21grep07:55:151
1996991311,2cyclictest27221-21diskmemload11:10:271
1996991310,2cyclictest7731-21cat09:55:241
199199139,3cyclictest3286-21cat07:10:190
199199132,1cyclictest0-21swapper/009:21:180
1991991310,2cyclictest521-21ls08:15:010
1991991310,2cyclictest11852-21grep07:30:140
199199130,3cyclictest0-21swapper/009:13:070
199199130,2cyclictest0-21swapper/012:29:190
199199130,1cyclictest0-21swapper/011:04:480
200299129,2cyclictest22760-21grep09:00:153
200299129,2cyclictest22582-21ps07:50:203
200299129,2cyclictest13986-21sed08:40:163
200299129,2cyclictest13986-21sed08:40:163
200299129,2cyclictest13535-21sed07:30:283
200299128,3cyclictest5619-21sort09:55:003
200299128,3cyclictest31675-21sort08:10:193
2002991211,1cyclictest0-21swapper/310:27:583
2002991211,1cyclictest0-21swapper/310:20:123
200299120,11cyclictest9726-21tr07:25:143
200099129,2cyclictest5113-21sed08:20:242
200099129,2cyclictest26243-21uniq10:45:162
200099129,2cyclictest21983-21grep08:55:242
200099129,2cyclictest14336-21egrep07:35:142
200099129,2cyclictest14095-21tail09:40:162
200099129,2cyclictest14016-21cron10:00:002
200099129,2cyclictest13918-21tail08:40:172
200099129,2cyclictest13918-21tail08:40:162
200099128,3cyclictest20836-21tr09:25:152
200099121,7cyclictest0-21swapper/212:38:442
200099121,10cyclictest7928-21/usr/sbin/munin10:15:142
200099120,10cyclictest0-21swapper/209:00:142
199699129,2cyclictest30096-21grep08:05:291
199699129,2cyclictest25366-21grep09:05:131
199699129,2cyclictest18725-21sort07:45:001
199699129,2cyclictest14946-21tail07:35:161
199699128,1cyclictest30750-21ssh10:28:511
199699121,1cyclictest9470-21munin-run10:55:001
199699120,4cyclictest141rcu_preempt11:40:511
199699120,2cyclictest0-21swapper/109:00:131
199199129,2cyclictest32765-21cut10:30:130
199199129,2cyclictest21944-21tr09:45:130
199199129,2cyclictest19560-21uniq07:45:180
199199129,2cyclictest17777-21grep07:40:200
199199129,2cyclictest13940-21grep08:40:160
199199129,2cyclictest13940-21grep08:40:160
199199129,2cyclictest13587-21tr09:40:140
199199128,2cyclictest16801-21sed08:45:210
1991991210,1cyclictest302522sleep011:25:340
199199120,6cyclictest0-21swapper/010:55:010
200299118,2cyclictest9169-21cron07:25:003
200299118,2cyclictest8111-21cut08:30:003
200299118,2cyclictest4852-21grep07:15:133
200299118,2cyclictest4475-21ssh10:32:063
200299118,2cyclictest29814-21ssh09:10:303
200299118,2cyclictest27071-21chrt09:08:213
200299118,2cyclictest2283-21wc08:15:213
200299118,2cyclictest15897-21grep07:35:263
200299118,2cyclictest15615-21cut08:45:133
200299118,2cyclictest12216-21grep11:15:173
200299118,2cyclictest11909-21chrt09:20:033
200099119,1cyclictest279552sleep211:25:092
200099118,2cyclictest913-21ssh10:10:262
200099118,2cyclictest5238-21sort07:15:152
200099118,2cyclictest4845-21ssh10:51:262
200099118,2cyclictest4373-21cat07:15:012
200099118,2cyclictest3693-21ssh12:26:592
200099118,2cyclictest32316-21sh09:50:402
200099118,2cyclictest31686-21ssh11:07:332
200099118,2cyclictest31582-21grep08:10:182
200099118,2cyclictest30532-21ssh09:30:242
200099118,2cyclictest28230-21sleep08:05:002
200099118,2cyclictest25525-21chrt07:56:282
200099118,2cyclictest25379-21tr09:05:132
200099118,2cyclictest24556-21ssh11:03:432
200099118,2cyclictest2106-21cat08:15:192
200099118,2cyclictest20866-21ssh10:41:432
200099118,2cyclictest16831-21tr07:40:132
200099118,2cyclictest15372-21ntp_states11:55:192
200099118,2cyclictest13356-21grep09:20:192
200099118,2cyclictest12046-21tr07:30:142
200099118,2cyclictest11112-21cat08:35:152
200099117,3cyclictest8710-21grep07:20:272
200099117,2cyclictest25413-21grep12:20:262
199699118,2cyclictest19741-21sed07:45:191
199699118,2cyclictest18584-21sort08:50:171
199699118,2cyclictest16075-21grep08:45:141
199699118,2cyclictest13536-21cat08:40:151
199699118,2cyclictest13536-21cat08:40:141
199699118,2cyclictest12053-21grep08:35:231
199699118,1cyclictest141rcu_preempt07:10:131
199699115,6cyclictest25-21ksoftirqd/110:55:071
199699111,4cyclictest141rcu_preempt10:35:141
199199119,1cyclictest23949-21tr07:55:130
199199118,2cyclictest7895-21cron08:30:000
199199118,2cyclictest5288-21grep07:15:150
199199118,2cyclictest4102-21cat09:15:160
199199118,2cyclictest30093-21grep09:30:200
199199118,2cyclictest24031-21sort10:25:010
199199118,2cyclictest17493-21tr10:40:160
199199118,2cyclictest14724-21ssh11:35:370
199199110,10cyclictest26528-21sh12:02:040
200299107,2cyclictest20994-21chrt07:49:153
200299107,2cyclictest19814-21taskset08:51:563
200299106,3cyclictest26340-21tr08:00:143
200299100,8cyclictest0-21swapper/307:55:133
200099107,2cyclictest9450-21dpkg07:25:122
200099107,2cyclictest30318-21chrt08:06:452
200099107,2cyclictest18781-21ls08:50:182
200099107,2cyclictest10375-21chrt08:32:542
199699108,1cyclictest9184-21expr08:30:161
199699108,1cyclictest28132-21taskset08:04:111
199699108,1cyclictest1830-21ls08:15:171
199699107,2cyclictest22740-21wc07:50:241
199699105,4cyclictest25-21ksoftirqd/108:25:131
199199108,1cyclictest9215-21ssh11:32:510
199199108,1cyclictest23077-21ls09:00:150
199199108,1cyclictest223062sleep008:58:280
199199108,1cyclictest19819-21taskset08:52:000
199199108,1cyclictest15997-21taskset07:36:180
199199108,1cyclictest12131-21grep08:35:240
199199107,2cyclictest6973-21sshd07:20:080
199199107,2cyclictest3708-21tr08:20:120
199199107,2cyclictest3235-21sh11:10:140
199199107,2cyclictest22973-21tail07:50:250
199199107,2cyclictest10223-21head07:25:160
199199105,4cyclictest1598-21/usr/sbin/munin08:15:150
199199100,9cyclictest25891-21smartctl09:05:160
19919986,1cyclictest278572sleep008:00:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional