You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 06:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Tue Apr 21, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
982721280,0sleep00-21swapper/021:26:060
2061728975,10sleep00-21swapper/019:06:540
2084328760,8sleep30-21swapper/319:09:503
2059428070,6sleep20-21swapper/219:06:362
2065227857,6sleep10-21swapper/119:07:191
205042470,0sleep20-21swapper/200:07:162
2093299290,2cyclictest141rcu_preempt00:17:251
20932992525,0cyclictest0-21swapper/121:53:341
20930992524,1cyclictest0-21swapper/000:15:140
20943992321,1cyclictest39-21ksoftirqd/321:17:173
2094399230,0cyclictest0-21swapper/323:24:583
2093899230,22cyclictest0-21swapper/200:29:332
2093899230,19cyclictest0-21swapper/223:10:512
2093299231,21cyclictest13739-21diskmemload21:33:271
2093299231,21cyclictest13739-21diskmemload21:33:271
2093899220,22cyclictest0-21swapper/223:44:042
2093899220,21cyclictest0-21swapper/223:30:312
20932992214,2cyclictest0-21swapper/123:31:541
2094399210,21cyclictest0-21swapper/300:01:543
202442210,0sleep20-21swapper/221:51:402
2094399200,1cyclictest0-21swapper/320:00:133
2094399200,19cyclictest20890-21sh22:50:173
2093899200,19cyclictest0-21swapper/222:57:442
20932992018,1cyclictest0-21swapper/122:35:241
20932992014,5cyclictest38150irq/131-snd_hda19:25:281
2093299200,1cyclictest0-21swapper/120:00:141
2094399190,2cyclictest0-21swapper/323:14:463
2093299190,18cyclictest0-21swapper/121:24:271
191952190,0sleep30-21swapper/321:31:543
191952190,0sleep30-21swapper/321:31:543
2094399180,3cyclictest0-21swapper/323:53:023
20943991714,2cyclictest39-21ksoftirqd/321:10:123
2094399170,2cyclictest0-21swapper/300:17:043
2094399170,1cyclictest0-21swapper/323:04:113
2094399170,17cyclictest0-21swapper/322:08:143
20938991712,2cyclictest0-21swapper/222:19:222
2093899170,17cyclictest0-21swapper/222:31:492
2093299176,6cyclictest225-21systemd-journal23:50:001
2093099171,5cyclictest12486-21hddtemp_smartct22:45:140
2094399167,6cyclictest39-21ksoftirqd/321:44:023
2094399162,13cyclictest0-21swapper/320:55:133
2094399160,15cyclictest29680-21kworker/u8:323:42:563
2093899160,1cyclictest0-21swapper/222:42:142
2093299160,14cyclictest0-21swapper/123:11:221
2093099160,3cyclictest0-21swapper/020:05:140
2094399159,6cyclictest39-21ksoftirqd/300:30:333
2094399154,4cyclictest141rcu_preempt21:48:073
20943991513,2cyclictest39-21ksoftirqd/322:28:323
20943991513,2cyclictest39-21ksoftirqd/321:35:233
2093899158,4cyclictest0-21swapper/200:10:182
20938991513,2cyclictest0-21swapper/200:24:452
2093899150,2cyclictest0-21swapper/223:08:512
2093899150,14cyclictest0-21swapper/221:49:572
2093299151,6cyclictest20825-21grep22:50:171
20932991514,1cyclictest38150irq/131-snd_hda23:18:391
20932991513,1cyclictest13739-21diskmemload21:11:511
2093299151,2cyclictest141rcu_preempt23:24:351
2093299151,1cyclictest20928-21cyclictest00:21:431
2093299150,15cyclictest0-21swapper/100:29:201
2093099151,13cyclictest0-21swapper/000:38:030
2093099150,14cyclictest0-21swapper/023:39:260
2094399149,4cyclictest141rcu_preempt00:27:073
2094399143,11cyclictest39-21ksoftirqd/323:18:453
2094399142,3cyclictest141rcu_preempt22:12:553
20943991414,0cyclictest0-21swapper/323:26:533
2094399141,1cyclictest0-21swapper/300:07:153
20943991410,3cyclictest11445-21ls19:55:213
2094399140,1cyclictest0-21swapper/322:59:473
2093899143,5cyclictest15755-21ssh00:05:002
20938991412,2cyclictest32-21ksoftirqd/223:58:162
20938991410,3cyclictest25516-21cut20:25:162
20938991410,3cyclictest2281-21cat19:35:262
20938991410,3cyclictest2281-21cat19:35:262
2093899140,9cyclictest0-21swapper/221:05:142
2093299149,4cyclictest434-21wc19:35:001
2093299146,7cyclictest26864-21apache222:05:261
2093299143,6cyclictest6575-21cron20:55:001
20932991411,2cyclictest12120-21sshd21:05:161
20932991410,3cyclictest16684-21head20:05:271
20932991410,1cyclictest0-21swapper/122:22:571
2093299140,13cyclictest0-21swapper/123:58:151
2093099147,4cyclictest537-21snmpd23:27:100
2093099146,4cyclictest455-21dbus-daemon20:20:000
2093099142,6cyclictest24102-21munin-run00:09:590
20930991411,2cyclictest26420-21grep20:25:260
20930991410,3cyclictest18401-21head00:25:170
20930991410,3cyclictest16345-21sed23:45:200
2093099140,10cyclictest0-21swapper/022:05:140
2094399139,3cyclictest20947-21cron23:10:003
2094399138,1cyclictest7140-21cron00:00:003
2094399132,3cyclictest225-21systemd-journal20:50:003
2094399132,2cyclictest30761-21apt-get23:35:123
2094399132,10cyclictest19401-21/usr/sbin/munin22:30:143
20943991310,2cyclictest32219-21tr20:40:133
20943991310,2cyclictest31364-21grep21:20:153
20943991310,2cyclictest22811-21cron23:50:003
20943991310,2cyclictest11693-21cut22:45:123
2094399130,1cyclictest0-21swapper/321:55:023
2094399130,13cyclictest0-21swapper/321:28:023
2093899139,3cyclictest22789-21sed19:10:272
2093899139,3cyclictest18692-21grep22:10:172
2093899131,4cyclictest225-21systemd-journal21:20:002
2093899131,3cyclictest0-21swapper/223:36:132
20938991312,1cyclictest0-21swapper/221:58:462
20938991311,1cyclictest8020-21chrt22:04:222
20938991310,2cyclictest5409-21grep23:00:172
20938991310,2cyclictest3803-21grep00:35:272
20938991310,2cyclictest28272-21grep20:30:232
20938991310,2cyclictest2539-21grep00:15:242
20938991310,2cyclictest20738-21head20:15:172
20938991310,2cyclictest19760-21cat20:15:002
2093899130,2cyclictest0-21swapper/221:27:082
2093899130,1cyclictest0-21swapper/221:35:082
2093299139,3cyclictest30800-21wc20:35:231
2093299139,3cyclictest26589-21wc00:10:261
2093299139,3cyclictest24160-21wc21:15:241
2093299138,2cyclictest27574-21tr22:15:201
2093299135,3cyclictest0-21swapper/121:45:111
20932991311,1cyclictest0-21swapper/122:02:481
20932991310,2cyclictest741-21head20:40:211
20932991310,2cyclictest3789-21head20:45:261
2093299130,13cyclictest0-21swapper/121:58:211
2093099139,3cyclictest18630-21grep21:50:260
2093099139,2cyclictest28304-21cat19:25:000
2093099131,1cyclictest13739-21diskmemload22:18:480
20930991310,2cyclictest5960-21cut23:00:220
20930991310,2cyclictest32457-21ssh23:55:150
20930991310,2cyclictest32206-21grep20:40:140
20930991310,2cyclictest28762-21basename22:55:140
20930991310,2cyclictest23658-21grep19:15:120
20930991310,2cyclictest16876-21cron00:25:000
2093099130,7cyclictest13739-21diskmemload23:45:000
75522120,2sleep00-21swapper/022:04:010
246632121,1sleep00-21swapper/000:10:080
2094399129,2cyclictest8485-21uniq19:50:183
2094399129,2cyclictest4759-21sed19:40:273
2094399129,2cyclictest28255-21head22:15:263
2094399129,2cyclictest27705-21sed19:20:263
2094399129,2cyclictest25102-21tr00:10:153
2094399129,2cyclictest2483-21hostname22:20:143
2094399129,2cyclictest2360-21sed19:35:263
2094399129,2cyclictest2360-21sed19:35:263
2094399129,2cyclictest23276-21cut23:30:183
2094399129,2cyclictest17111-21sort00:25:003
2094399129,2cyclictest141rcu_preempt22:01:413
2094399129,2cyclictest11851-21tr21:05:153
2094399128,3cyclictest24233-21cut19:15:163
2094399121,6cyclictest0-21swapper/300:35:143
2093899129,2cyclictest9466-21grep19:50:282
2093899129,2cyclictest4815-21grep19:40:272
2093899129,2cyclictest3730-21grep20:45:262
2093899129,2cyclictest30390-21ssh23:15:162
2093899129,2cyclictest2850-21ssh22:40:002
2093899129,2cyclictest26665-21grep00:30:162
2093899129,2cyclictest11931-21ssh22:45:122
2093899127,4cyclictest23311-21acpi23:50:122
2093899126,2cyclictest225-21systemd-journal19:25:002
2093299129,2cyclictest9964-21tail21:00:161
2093299129,2cyclictest5748-21grep19:45:121
2093299129,2cyclictest4333-21grep19:40:231
2093299129,2cyclictest27892-21grep20:30:201
2093299129,2cyclictest27639-21sed19:20:261
2093299129,2cyclictest24774-21sed19:15:211
2093299129,2cyclictest16844-21head00:05:171
2093299129,2cyclictest1611-21sed21:40:281
2093299129,2cyclictest15056-21tail23:05:261
2093299129,2cyclictest13472-21sed22:45:231
2093299129,2cyclictest10906-21sort19:55:161
2093299128,3cyclictest4680-21cut23:00:141
2093299128,3cyclictest20316-21grep22:30:201
2093299128,3cyclictest18053-21grep22:10:141
2093299128,3cyclictest17598-21ssh22:29:151
2093299127,3cyclictest38150irq/131-snd_hda23:25:101
20932991210,1cyclictest0-21swapper/100:35:201
2093299120,11cyclictest3470-21ssh23:37:541
2093299120,11cyclictest2810-21cron19:40:001
2093299120,11cyclictest2810-21cron19:40:001
2093099129,2cyclictest6074-21tr23:20:150
2093099129,2cyclictest5465-21wc20:50:170
2093099129,2cyclictest31208-21grep20:35:270
2093099129,2cyclictest26081-21grep00:30:130
2093099129,2cyclictest25694-21grep21:55:160
2093099129,2cyclictest22655-21grep20:20:150
2093099129,2cyclictest18177-21cut22:10:150
2093099129,2cyclictest12075-21tr22:25:230
2093099129,2cyclictest11866-21grep19:55:260
2093099128,3cyclictest30548-21chrt23:15:170
2093099128,3cyclictest16092-21grep21:10:240
2093099128,3cyclictest13376-21head20:00:170
2093099128,2cyclictest27654-21cut22:35:170
2093099123,7cyclictest13739-21diskmemload23:05:070
2094399119,1cyclictest28323-21grep22:35:203
2094399118,2cyclictest9613-21expr21:00:143
2094399118,2cyclictest8597-21ssh22:43:003
2094399118,2cyclictest5951-21expr19:45:143
2094399118,2cyclictest4721-21tr20:50:133
2094399118,2cyclictest28588-21sort19:25:153
2094399118,2cyclictest21994-21sort19:10:183
2094399118,2cyclictest16392-21sh21:50:023
2094399118,2cyclictest15355-21cat20:05:163
2093899118,2cyclictest8187-21seq23:20:462
2093899118,2cyclictest5207-21tail20:50:152
2093899118,2cyclictest4578-21ssh22:20:332
2093899118,2cyclictest32754-21taskset19:33:422
2093899118,2cyclictest319352sleep220:40:122
2093899118,2cyclictest30945-21ssh21:20:082
2093899118,2cyclictest29649-21wc19:25:222
2093899118,2cyclictest18778-21ssh23:46:492
2093899118,2cyclictest18742-21ssh21:31:322
2093899118,2cyclictest18742-21ssh21:31:322
2093899118,2cyclictest17201-21ssh21:11:112
2093899118,2cyclictest16298-21ssh23:25:372
2093899118,2cyclictest10658-21ssh22:05:182
2093899117,2cyclictest21917-21grep22:50:262
2093899110,10cyclictest24931-21grep19:15:212
2093899110,10cyclictest15471-21ssh22:27:352
2093899110,10cyclictest13877-21ps20:00:222
2093299118,2cyclictest8573-21sed23:40:261
2093299118,2cyclictest27924-21sed00:30:251
2093299118,2cyclictest19504-21chrt20:12:421
2093299117,3cyclictest7098-21cut20:55:141
2093299117,3cyclictest25108-21wc23:50:261
2093299114,5cyclictest26876-21apache221:25:131
2093299110,10cyclictest9464-21cat00:00:291
2093299110,10cyclictest7024-21ssh22:41:451
2093299110,10cyclictest23439-21ssh21:35:021
2093099118,2cyclictest8374-21ssh22:42:490
2093099118,2cyclictest7108-21tr20:55:140
2093099118,2cyclictest6078-21sort19:45:150
2093099118,2cyclictest5489-21chrt22:21:140
2093099118,2cyclictest31902-21head21:20:170
2093099118,2cyclictest28453-21wget19:25:150
2093099118,2cyclictest27781-21cut20:30:190
2093099118,2cyclictest27356-21ssh21:17:320
2093099118,2cyclictest26489-21ssh21:36:030
2093099118,2cyclictest23927-21ssh22:32:410
2093099118,2cyclictest23633-21tr23:50:160
2093099118,2cyclictest22544-21cat23:10:190
2093099118,2cyclictest22347-21ssh23:30:120
2093099118,2cyclictest21600-21expr19:10:160
2093099118,2cyclictest1298-21which21:40:240
2093099118,2cyclictest11777-21ssh21:46:460
2093099117,3cyclictest11350-21cron21:05:000
2093099112,5cyclictest9725-21grep00:00:300
2093099111,1cyclictest0-21swapper/022:50:170
2093099110,10cyclictest17633-21ssh21:30:400
2093099110,10cyclictest17633-21ssh21:30:390
128572110,0sleep00-21swapper/021:05:220
2094399108,1cyclictest32577-21seq19:31:263
2094399108,1cyclictest30188-21tune2fs20:35:173
2094399108,1cyclictest28044-21cat20:30:213
2094399108,1cyclictest26125-21sensors20:25:233
2094399108,1cyclictest19959-21apache_volume20:15:133
2094399107,2cyclictest23105-21head20:20:183
2094399100,9cyclictest19496-21chrt20:12:363
2093899108,1cyclictest7140-21cut20:55:132
2093899108,1cyclictest6628-21ntpq19:45:192
2093899108,1cyclictest31946-21ssh21:40:042
2093899108,1cyclictest31010-21cat20:35:262
2093899108,1cyclictest22569-21cat20:20:152
2093899108,1cyclictest11087-21cat19:55:182
2093299108,1cyclictest25359-21expr20:25:161
2093299108,1cyclictest241702sleep120:21:151
2093299108,1cyclictest21231-21grep19:10:151
2093299107,2cyclictest9275-21expr19:50:271
2093299107,2cyclictest284002sleep122:55:031
2093099108,1cyclictest1536-21ls19:35:170
2093099108,1cyclictest1536-21ls19:35:160
2093099107,2cyclictest2170-21cut20:45:140
2093099106,3cyclictest9965-21apt19:55:000
2093099105,2cyclictest225-21systemd-journal19:40:150
209389997,1cyclictest15007-21aten2_r7power_c20:05:132
209329997,1cyclictest218012sleep120:16:391
209309996,2cyclictest32206-21sort19:30:250
209309996,2cyclictest18425-21if_enp0s31f620:10:180
209389986,1cyclictest10053-21cat21:00:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional