You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 03:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Fri Sep 04, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1842328875,9sleep00-21swapper/019:06:460
1865328661,9sleep20-21swapper/219:09:442
1851327958,6sleep10-21swapper/119:07:551
1842127958,6sleep30-21swapper/319:06:433
1876999270,27cyclictest0-21swapper/222:08:202
18765992626,0cyclictest0-21swapper/100:24:461
1876199250,2cyclictest0-21swapper/000:17:070
18765992323,0cyclictest0-21swapper/121:10:281
1876199230,1cyclictest0-21swapper/023:05:340
1876999221,20cyclictest0-21swapper/200:15:232
1876199222,20cyclictest0-21swapper/021:41:230
1877299210,1cyclictest0-21swapper/323:15:143
1876599210,1cyclictest0-21swapper/100:05:501
18772992018,2cyclictest0-21swapper/322:24:513
1876999200,17cyclictest0-21swapper/222:28:582
1876199209,9cyclictest13-21ksoftirqd/023:33:530
18772991918,1cyclictest0-21swapper/322:45:373
18772991917,2cyclictest0-21swapper/322:26:023
1877299191,17cyclictest26758-21rm22:35:253
1876599191,17cyclictest6179-21cstates19:50:161
1876599191,17cyclictest30580-21apt-get21:40:151
1876599190,18cyclictest14927-21ssh23:26:251
99032180,3sleep29908-21unixbench_multi19:55:282
18772991816,1cyclictest0-21swapper/322:55:473
18772991814,3cyclictest17315-21kworker/3:100:27:323
1876999184,2cyclictest225-21systemd-journal00:00:002
1876999180,1cyclictest0-21swapper/222:04:552
1876999180,1cyclictest0-21swapper/221:38:452
1876999180,16cyclictest0-21swapper/200:25:402
1876999180,0cyclictest0-21swapper/223:46:032
1876599180,0cyclictest0-21swapper/121:45:231
1876199181,2cyclictest0-21swapper/022:57:240
18769991716,1cyclictest0-21swapper/223:43:592
1876999170,16cyclictest0-21swapper/221:27:142
1876999170,15cyclictest0-21swapper/220:40:022
1876599170,4cyclictest10813-21kworker/u8:022:27:261
18761991716,1cyclictest0-21swapper/022:18:110
18761991716,1cyclictest0-21swapper/021:38:450
18761991712,1cyclictest13-21ksoftirqd/023:45:220
1876999162,10cyclictest2331-21apt-get23:00:142
18769991615,1cyclictest55650irq/127-enp2s0-00:00:202
18769991614,2cyclictest32-21ksoftirqd/200:32:442
18769991613,2cyclictest32-21ksoftirqd/222:59:592
1876999161,12cyclictest6345-21/usr/sbin/munin21:45:152
18765991612,2cyclictest25-21ksoftirqd/121:00:151
1876599161,14cyclictest12192-21diskmemload23:54:211
1876599160,4cyclictest11742-21missed_timers23:05:181
18772991513,1cyclictest12192-21diskmemload00:00:283
18772991511,1cyclictest0-21swapper/322:31:153
18772991510,4cyclictest4204-21kworker/3:223:35:133
1876999155,1cyclictest455-21dbus-daemon23:20:002
1876999153,8cyclictest25938-21tail22:35:182
1876999150,1cyclictest0-21swapper/200:06:402
1876999150,14cyclictest0-21swapper/223:38:012
18765991514,1cyclictest0-21swapper/100:28:021
1876599150,1cyclictest0-21swapper/122:05:151
1876199151,1cyclictest141rcu_preempt22:09:210
1876199150,10cyclictest0-21swapper/021:15:020
18772991412,1cyclictest2347-21ssh23:00:143
18772991410,3cyclictest31134-21head20:40:163
18772991410,3cyclictest22615-21sh21:54:593
18772991410,3cyclictest12506-21head23:25:173
1876999142,5cyclictest12192-21diskmemload21:52:412
1876999141,4cyclictest0-21swapper/221:55:042
1876999141,13cyclictest23576-21kworker/u8:021:21:422
1876999141,13cyclictest12192-21diskmemload22:14:352
1876999140,13cyclictest0-21swapper/222:42:192
1876999140,11cyclictest0-21swapper/223:51:302
1876599148,2cyclictest26095-21grep22:15:261
1876599142,11cyclictest12192-21diskmemload22:57:251
18765991414,0cyclictest0-21swapper/100:03:411
18765991412,2cyclictest25-21ksoftirqd/122:45:431
18765991412,1cyclictest0-21swapper/121:22:271
18765991411,2cyclictest12192-21diskmemload00:19:421
18765991410,3cyclictest31534-21tr20:40:191
18765991410,3cyclictest23524-21sleep19:20:001
18765991410,3cyclictest19751-21grep22:50:321
1876199146,7cyclictest4850-21awk20:55:010
1876199146,5cyclictest0-21swapper/021:12:000
1876199142,8cyclictest3271-21/usr/sbin/munin23:20:170
18761991413,1cyclictest0-21swapper/020:34:370
1876199140,8cyclictest12192-21diskmemload22:40:280
1876199140,1cyclictest0-21swapper/022:28:580
1877299137,6cyclictest0-21swapper/322:04:353
1877299134,5cyclictest225-21systemd-journal20:00:563
18772991313,0cyclictest0-21swapper/323:10:413
18772991312,1cyclictest0-21swapper/321:45:363
18772991311,2cyclictest4204-21kworker/3:222:52:433
18772991310,2cyclictest28282-21grep20:35:163
18772991310,2cyclictest26008-21wc19:25:003
18772991310,2cyclictest14503-21wc20:05:223
18772991310,2cyclictest12192-21diskmemload23:40:183
18772991310,1cyclictest6549-21kworker/3:021:35:433
1877299130,8cyclictest0-21swapper/322:15:123
1877299130,12cyclictest0-21swapper/300:19:023
18769991311,2cyclictest0-21swapper/221:31:172
18769991310,2cyclictest32-21ksoftirqd/219:40:222
18769991310,2cyclictest2963-21tr20:50:142
1876999130,10cyclictest0-21swapper/222:52:472
1876599139,3cyclictest1517-21grep22:20:221
1876599132,2cyclictest12192-21diskmemload21:55:221
18765991312,1cyclictest38150irq/131-snd_hda21:35:081
18765991312,1cyclictest12192-21diskmemload22:44:011
18765991312,1cyclictest0-21swapper/123:18:151
18765991312,1cyclictest0-21swapper/123:10:171
18765991310,2cyclictest5125-21grep19:45:271
18765991310,2cyclictest28051-21tr20:35:151
18765991310,2cyclictest27532-21apt20:35:011
18765991310,2cyclictest20409-21tr23:30:191
18765991310,2cyclictest16711-21sed21:50:301
1876199139,4cyclictest23035-21sh22:14:330
1876199135,7cyclictest10098-21munin-run23:05:000
1876199133,8cyclictest0-21swapper/021:53:420
1876199131,2cyclictest0-21swapper/000:25:410
1876199131,11cyclictest5494-21/usr/sbin/munin00:00:250
18761991310,2cyclictest425-21awk22:00:210
1876199130,12cyclictest0-21swapper/023:43:590
1877299129,2cyclictest7687-21tr22:05:173
1877299129,2cyclictest29183-21cat21:19:593
1877299129,2cyclictest25185-21grep21:55:273
1877299129,2cyclictest24345-21grep00:10:253
1877299129,2cyclictest20557-21cron21:15:003
1877299129,2cyclictest1687-21chrt21:22:283
1877299128,4cyclictest4204-21kworker/3:223:50:233
1877299128,3cyclictest30942-21tr23:55:223
1877299128,3cyclictest1088-21grep20:45:173
1877299128,1cyclictest0-21swapper/322:41:323
1877299127,3cyclictest11355-21cat23:24:593
1877299124,4cyclictest452-21systemd-logind00:07:253
18772991210,2cyclictest176502sleep323:09:233
1876999129,2cyclictest7908-21cron22:25:012
1876999129,2cyclictest32-21ksoftirqd/223:05:122
1876999129,2cyclictest31733-21cut19:35:152
1876999127,4cyclictest12192-21diskmemload23:24:202
1876999123,8cyclictest32-21ksoftirqd/200:35:282
1876999123,5cyclictest30678-21cron19:34:592
18769991211,1cyclictest0-21swapper/222:45:132
1876999120,9cyclictest0-21swapper/223:34:262
1876999120,12cyclictest0-21swapper/221:44:592
1876599129,3cyclictest18756-21cyclictest23:40:481
1876599129,2cyclictest8526-21grep19:55:151
1876599129,2cyclictest669-21sleep19:40:001
1876599129,2cyclictest5862-21head20:55:171
1876599129,2cyclictest3371-21ssh23:20:131
1876599129,2cyclictest31880-21ssh00:35:161
1876599129,2cyclictest25537-21grep19:20:261
1876599129,2cyclictest13485-21grep23:45:171
1876599129,2cyclictest13325-21basename20:05:151
1876599128,3cyclictest29332-21grep19:30:181
1876599128,1cyclictest20946-21taskset19:13:101
1876599127,4cyclictest26742-21ssh00:31:331
18765991210,1cyclictest23241-21grep20:25:151
1876599120,2cyclictest0-21swapper/100:12:041
1876199129,3cyclictest13-21ksoftirqd/023:52:530
1876199129,2cyclictest821-21ssh00:35:240
1876199129,2cyclictest7672-21cut21:45:190
1876199129,2cyclictest32292-21ssh22:39:300
1876199129,2cyclictest31346-21ssh23:16:590
1876199129,2cyclictest29872-21grep23:35:220
1876199129,2cyclictest25325-21rm23:13:470
1876199129,2cyclictest25067-21grep21:55:260
1876199129,2cyclictest24688-21ssh22:54:160
1876199129,2cyclictest20445-21rm22:31:450
1876199129,2cyclictest14254-21grep21:30:190
1876199129,2cyclictest14017-21head20:05:180
1876199129,2cyclictest13471-21ssh22:47:050
1876199127,3cyclictest0-21swapper/021:27:140
1876199120,7cyclictest0-21swapper/023:27:100
1877299118,2cyclictest9959-21sed19:55:283
1877299118,2cyclictest9225-21taskset21:00:293
1877299118,2cyclictest7753-21taskset19:52:033
1877299118,2cyclictest5879-21grep20:55:173
1877299118,2cyclictest5658-21cut21:25:163
1877299118,2cyclictest31883-21grep00:35:163
1877299118,2cyclictest31562-21cat21:40:213
1877299118,2cyclictest27039-21ssh00:31:473
1877299118,2cyclictest26951-21grep19:25:183
1877299118,2cyclictest26874-21cat20:30:253
1877299118,2cyclictest23248-21ssh23:31:223
1877299118,2cyclictest20832-21tr20:20:153
1877299118,2cyclictest1747-21cut19:40:163
1877299118,2cyclictest16720-21cat20:10:183
1877299118,2cyclictest16701-21ssh22:10:203
1877299118,2cyclictest16216-21ssh21:30:453
1877299118,2cyclictest12762-21aten2_r7power_c23:45:123
1877299117,3cyclictest3484-21grep20:50:153
1877299115,4cyclictest12196-21apt21:10:003
1877299114,3cyclictest225-21systemd-journal19:20:003
1877299113,4cyclictest225-21systemd-journal19:50:003
1877299112,6cyclictest0-21swapper/300:20:153
1876999119,2cyclictest27902-21awk20:35:152
1876999119,2cyclictest25173-21tr19:20:232
1876999119,2cyclictest25074-21rm21:16:562
1876999119,2cyclictest22319-21sh00:10:102
1876999119,2cyclictest13954-21ssh23:25:392
1876999119,1cyclictest5880-21sed20:55:172
1876999119,1cyclictest23341-21taskset19:18:202
1876999119,1cyclictest21522-21ssh23:10:522
1876999119,1cyclictest18320-21grep22:30:262
1876999118,2cyclictest8753-21sh00:20:312
1876999118,2cyclictest7771-21tr21:00:162
1876999118,2cyclictest4474-21wc19:45:202
1876999118,2cyclictest27243-21cat19:25:202
1876999118,2cyclictest24698-21grep20:25:272
1876999118,2cyclictest19200-21wc20:15:202
1876999118,2cyclictest19200-21wc20:15:192
1876999118,2cyclictest17090-21wc20:10:212
1876999118,2cyclictest14725-21wc20:05:262
1876999114,4cyclictest21804-21pmu-power20:20:232
1876999113,7cyclictest0-21swapper/221:13:132
1876999113,6cyclictest28814-21sh22:17:292
1876999111,4cyclictest0-21swapper/219:10:132
1876999110,1cyclictest141rcu_preempt20:00:152
1876599119,1cyclictest3826-21strings20:50:191
1876599118,2cyclictest5883-21expr21:25:171
1876599118,2cyclictest31971-21sh23:55:411
1876599118,2cyclictest30915-21ssh23:35:471
1876599118,2cyclictest20356-21sh21:33:591
1876599118,2cyclictest1870-21ssh23:00:001
1876599118,2cyclictest1425-21ssh22:00:411
1876599118,1cyclictest24957-21sh22:35:061
1876599118,1cyclictest11789-21ntpq20:00:201
1876599113,3cyclictest225-21systemd-journal21:17:451
1876599111,1cyclictest141rcu_preempt22:13:371
18765991110,1cyclictest0-21swapper/122:30:411
1876199119,2cyclictest2160-21ssh23:57:450
1876199119,2cyclictest20176-21open_inodes19:10:200
1876199119,1cyclictest8561-21seq19:55:150
1876199119,1cyclictest28525-21head20:35:180
1876199119,1cyclictest23471-21df20:25:160
1876199119,1cyclictest22817-21date20:25:010
1876199119,1cyclictest22813-21ssh00:10:150
1876199119,1cyclictest15953-21aten2_r7power_v20:10:130
1876199118,2cyclictest8882-21ssh00:20:380
1876199118,2cyclictest8259-21head21:00:190
1876199118,2cyclictest537-21snmpd00:06:010
1876199118,2cyclictest32519-21ssh22:20:120
1876199118,2cyclictest31654-21grep19:35:150
1876199118,2cyclictest23978-21cat19:20:140
1876199118,2cyclictest23220-21ssh00:30:090
1876199115,3cyclictest30400-21grep21:20:170
1876199114,5cyclictest21374-21apt-get19:15:120
1876199110,5cyclictest225-21systemd-journal19:35:000
1876199110,0cyclictest0-21swapper/020:40:030
1877299107,2cyclictest30177-21ls19:30:283
1877299107,2cyclictest23625-21grep20:25:163
1877299104,2cyclictest225-21systemd-journal20:15:013
1877299104,2cyclictest225-21systemd-journal20:15:013
1877299100,9cyclictest31185-21expr19:35:133
1876999108,1cyclictest272422sleep220:31:302
1876999107,2cyclictest6164-21tr19:50:162
1876999107,2cyclictest1832-21wc20:45:262
1876999106,3cyclictest11477-21smart_sda21:05:252
1876599108,1cyclictest20227-21taskset20:18:571
1876599108,1cyclictest20227-21taskset20:18:561
1876599108,1cyclictest10474-21cat21:05:141
1876599107,2cyclictest27682-21wc19:25:261
1876599107,2cyclictest22586-21sshd20:23:291
1876599107,2cyclictest17081-21proc_pri20:10:211
1876599106,3cyclictest2191-21taskset20:46:421
1876599100,9cyclictest2438-21processes19:40:231
1876199109,1cyclictest9902-21apt-config21:04:590
1876199109,1cyclictest4982-21cat19:45:260
1876199108,1cyclictest6611-21grep19:50:180
1876199108,1cyclictest543-21cat20:45:150
1876199108,1cyclictest26257-21cut19:25:140
1876199108,1cyclictest18978-21head20:15:170
1876199108,1cyclictest18978-21head20:15:170
187729996,2cyclictest19061-21chrt19:10:133
187619998,1cyclictest11813-21sh20:00:210
187619998,1cyclictest1179-21basename19:40:140
187619997,1cyclictest5858-21grep20:55:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional