You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 13:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Sat Sep 05, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
105632990,0sleep50-21swapper/521:18:235
136692340,0sleep50-21swapper/523:10:145
3322992416,3cyclictest765-21turbostat22:45:002
331499196,9cyclictest19607-21sh23:13:450
331499195,10cyclictest1417-21sh22:26:050
333199185,9cyclictest1107-21ssh21:12:554
333199185,9cyclictest10365-21sh00:21:514
333199184,10cyclictest15981-21taskset20:44:414
331499185,9cyclictest25364-21sh00:11:580
331499185,9cyclictest14959-21sh22:34:340
331499181,9cyclictest774-21runrttasks21:46:210
331499180,15cyclictest1570-21chrt23:21:120
331499180,14cyclictest3757-21sh00:00:030
331499180,14cyclictest19644-21sh22:18:320
331499180,10cyclictest6485-21runrttasks00:20:120
333199175,8cyclictest7363-21ssh22:30:124
333199175,7cyclictest27592-21sh00:31:424
333199175,7cyclictest23796-21rm23:34:384
333199174,8cyclictest5751-21grep21:15:174
333199174,8cyclictest321-21systemd-journal20:04:414
333199174,8cyclictest19552-21ssh00:09:034
333199173,10cyclictest20297-21/usr/sbin/munin23:50:214
3327991710,5cyclictest22198-21if_enp6s000:10:173
331499175,9cyclictest7173-21sh22:48:170
331499175,9cyclictest7173-21sh22:48:170
331499175,9cyclictest11035-21sh22:13:560
331499170,9cyclictest30647-21sh00:33:570
331499170,9cyclictest25461-21sh22:21:220
331499170,11cyclictest24087-21sed21:25:250
333199166,8cyclictest15555-21sh22:15:374
333199165,8cyclictest26039-21ssh21:26:484
333199165,6cyclictest352150latency_hist23:05:004
333199164,8cyclictest8151-21ntpq20:25:194
333199164,8cyclictest7853-21tune2fs23:25:164
333199164,8cyclictest6837-21ssh00:38:504
333199164,8cyclictest3863-21taskset21:32:494
333199164,8cyclictest25485-21cut22:40:134
333199164,8cyclictest22941-21expr22:20:164
333199164,7cyclictest23701-21ssh22:39:304
333199163,9cyclictest16391-21sh21:58:294
333199163,8cyclictest20187-21ssh21:23:564
3331991613,2cyclictest31221-21cpuspeed_turbos22:25:124
332799163,9cyclictest21850-21latency_hist21:25:013
3327991612,3cyclictest697-21polkitd21:30:193
332799160,9cyclictest15127-21ssh23:29:443
332299163,9cyclictest14366-21proc_pri23:10:192
331499160,13cyclictest774-21runrttasks20:09:000
331499160,10cyclictest774-21runrttasks19:46:360
331499160,10cyclictest10284-21ssh21:36:060
333199156,8cyclictest26072-21awk23:35:184
333199156,7cyclictest711-21snmpd20:33:204
333199155,8cyclictest774-21runrttasks20:59:374
333199155,8cyclictest774-21runrttasks20:59:374
333199155,8cyclictest7393-21ssh21:35:104
333199155,8cyclictest711-21snmpd19:43:564
333199155,8cyclictest3498-21ssh21:50:294
333199155,8cyclictest3310-40mandb00:00:024
333199155,8cyclictest23833-21sh22:02:264
333199155,8cyclictest22373-21taskset19:45:314
333199155,8cyclictest12458-21kernelversion20:35:174
333199154,7cyclictest4217-21ssh22:10:084
333199154,6cyclictest10039-21/usr/sbin/munin22:50:194
333199154,6cyclictest10039-21/usr/sbin/munin22:50:194
333199153,8cyclictest6189-21/usr/sbin/munin23:05:234
333199153,8cyclictest5311-21ssh23:42:004
333199153,8cyclictest2775-21sh00:17:454
333199153,8cyclictest27463-21ssh23:55:104
333199153,8cyclictest26393-21ps21:45:184
333199153,8cyclictest22943-21idleruntime23:15:154
333199153,8cyclictest22122-21sh22:56:224
333199153,8cyclictest22005-21perl00:10:164
333199153,8cyclictest21262-21ssh21:42:364
333199153,8cyclictest13492-21ssh23:10:134
333199153,8cyclictest11830-21users23:45:234
333199153,8cyclictest11473-21diskstats19:25:134
333199153,8cyclictest1098-21ssh22:45:034
333199153,8cyclictest1098-21ssh22:45:034
332799151,10cyclictest4704-21sh00:37:163
332799151,10cyclictest32365-21wget20:10:143
332799151,10cyclictest31666-21rm22:43:553
332799151,10cyclictest2639-21/usr/sbin/munin22:45:193
332799151,10cyclictest2639-21/usr/sbin/munin22:45:193
332799151,10cyclictest24408-21latency_hist22:40:003
332299154,8cyclictest11767-21sed22:50:212
332299154,8cyclictest11767-21sed22:50:212
3322991511,3cyclictest12815-21cron21:20:012
332299151,10cyclictest6257-21runrttasks20:22:062
332299151,10cyclictest25913-21idleruntime21:45:152
331899154,8cyclictest11248-21cpu19:25:121
331899151,10cyclictest9884-21ssh21:35:481
331899151,10cyclictest20438-21/usr/sbin/munin22:00:191
331899151,10cyclictest11946-21ssh22:32:221
331899150,11cyclictest7816-21ssh21:53:371
331499156,7cyclictest24160-21ssh22:57:510
331499156,7cyclictest15526-21rm22:52:580
331499156,7cyclictest15526-21rm22:52:580
331499155,8cyclictest31594-21/usr/sbin/munin00:15:240
331499151,10cyclictest774-21runrttasks00:35:190
331499151,10cyclictest17868-21/usr/sbin/munin22:35:230
331499150,12cyclictest18929-21rm22:00:030
331499150,11cyclictest774-21runrttasks23:43:090
333199146,6cyclictest25317-21munin-run21:05:004
333199146,6cyclictest25317-21munin-run21:05:004
333199145,7cyclictest5687-21sendmail_mailtr20:20:184
333199145,7cyclictest32153-21/usr/sbin/munin23:20:184
333199145,5cyclictest14175-21/usr/sbin/munin19:30:164
333199144,8cyclictest9997-21cat19:20:254
333199144,6cyclictest32671latency_hist19:10:014
333199143,8cyclictest23513-21perl19:50:144
333199143,8cyclictest19340-21/usr/sbin/munin20:50:154
333199143,7cyclictest27897-21/usr/sbin/munin22:05:234
333199143,6cyclictest25318-21/usr/sbin/munin19:55:114
333199143,6cyclictest2500-21diskstats20:15:134
333199143,6cyclictest16446-21cat19:35:164
333199141,9cyclictest16466-21/usr/sbin/munin00:25:204
332799146,6cyclictest16588-21tune2fs23:30:133
332799144,7cyclictest25808-21kworker/u24:4+nfsiod23:00:493
332799143,8cyclictest28694-21tr20:00:183
332799141,9cyclictest27846-21ssh00:31:533
332799141,9cyclictest2031-21sh22:26:313
332799141,9cyclictest19002-21ssh21:40:563
332799141,10cyclictest27443-21chrt23:55:093
332799141,10cyclictest21563-21gltestperf20:55:143
332799141,10cyclictest21563-21gltestperf20:55:143
332799140,10cyclictest9004-21cut19:20:153
332799140,10cyclictest8698-21ssh23:07:113
332799140,10cyclictest7744-21sh21:16:203
332799140,10cyclictest7148-21grep19:15:203
332799140,10cyclictest7082-21/usr/sbin/munin20:25:213
332799140,10cyclictest6366-21chrt20:23:363
332799140,10cyclictest32420-21sh00:15:443
332799140,10cyclictest30226-21rm21:10:313
332799140,10cyclictest30007-21runrttasks22:05:333
332799140,10cyclictest29944-21/usr/sbin/munin20:05:113
332799140,10cyclictest28269-21sh21:28:243
332799140,10cyclictest27406-21rm21:45:403
332799140,10cyclictest25609-21/usr/sbin/munin21:05:123
332799140,10cyclictest25380-21chrt23:16:203
332799140,10cyclictest21696-21ssh23:51:103
332799140,10cyclictest21139-21ssh23:14:523
332799140,10cyclictest19579-21grep22:00:143
332799140,10cyclictest18755-21ssh22:55:073
332799140,10cyclictest17924-21apt-key19:40:013
332799140,10cyclictest16379-21grep20:45:113
332799140,10cyclictest16091-21ssh22:16:003
332799140,10cyclictest14620-21sh00:05:263
332799140,10cyclictest10911-21apt-config19:25:003
332799140,10cyclictest10744-21ssh22:31:303
332799140,10cyclictest10611-21sh22:13:383
332299141,9cyclictest774-21runrttasks23:20:082
332299141,9cyclictest774-21runrttasks22:56:322
332299141,9cyclictest7684-21cpuspeed_turbos21:35:132
332299141,9cyclictest4242-21hddtemp_smartct19:10:132
332299141,9cyclictest24892-21taskset22:20:572
332299141,9cyclictest22444-21diskmemload22:38:352
332299141,9cyclictest19276-21fschecks_time23:50:132
332299141,9cyclictest17705-21sh21:40:212
332299141,9cyclictest15851-21sh23:30:022
332299141,9cyclictest10167-21kernelversion20:30:142
332299141,10cyclictest11144-21runrttasks23:26:482
3322991410,3cyclictest4519-21cpuspeed_turbos22:10:122
3322991410,3cyclictest321-21systemd-journal23:05:012
332299140,10cyclictest9785-21sh23:45:042
332299140,10cyclictest8511-21/usr/sbin/munin19:20:202
332299140,10cyclictest7498-21sh22:48:312
332299140,10cyclictest7498-21sh22:48:312
332299140,10cyclictest5425-21runrttasks21:51:522
332299140,10cyclictest4244-21sh00:36:552
332299140,10cyclictest3593-21ssh21:14:422
332299140,10cyclictest32153-21sh23:02:152
332299140,10cyclictest28090-21ssh21:28:162
332299140,10cyclictest26710-21ssh23:17:192
332299140,10cyclictest25815-21ssh00:30:242
332299140,10cyclictest23665-21df_inode21:00:122
332299140,10cyclictest23665-21df_inode21:00:112
332299140,10cyclictest21038-21sed00:10:002
332299140,10cyclictest19822-21ssh22:18:412
332299140,10cyclictest16814-21fschecks_time20:45:152
332299140,10cyclictest1564-21ssh00:16:522
332299140,10cyclictest12753-21sh21:55:492
332299140,10cyclictest12746-21/usr/sbin/munin20:35:182
332299140,10cyclictest12135-21rm22:32:302
331899144,8cyclictest683-21dbus-daemon22:15:011
331899143,8cyclictest751-21postgres20:40:431
331899141,9cyclictest745-21postgres20:22:431
331899141,9cyclictest27416-21diskmemload21:27:461
331899141,9cyclictest2445-21ssh00:35:381
331899141,9cyclictest19851-21timerwakeupswit19:40:231
331899141,9cyclictest15545-21sh22:52:591
331899141,9cyclictest15545-21sh22:52:591
331899141,9cyclictest1087-21runrttasks20:10:221
3318991411,2cyclictest22798-21gltestperf23:15:141
3318991410,3cyclictest10218-21cpuspeed_turbos23:45:121
331899140,10cyclictest8601-21sh22:12:101
331899140,10cyclictest8299-21mailstats20:25:201
331899140,10cyclictest774-21runrttasks23:50:321
331899140,10cyclictest774-21runrttasks21:41:061
331899140,10cyclictest7645-21ssh00:01:431
331899140,10cyclictest7398-21ssh21:16:051
331899140,10cyclictest7335-21/usr/sbin/munin23:25:141
331899140,10cyclictest6863-21sh21:35:001
331899140,10cyclictest6511-21/usr/sbin/munin00:20:181
331899140,10cyclictest4406-21runrttasks23:41:201
331899140,10cyclictest4205-21chrt22:46:061
331899140,10cyclictest4205-21chrt22:46:061
331899140,10cyclictest31928-21ssh23:02:051
331899140,10cyclictest31523-21ssh00:15:221
331899140,10cyclictest29386-21rm21:10:181
331899140,10cyclictest27712-21sh22:05:031
331899140,10cyclictest27451-21runrttasks21:07:571
331899140,10cyclictest25851-21sh21:45:141
331899140,10cyclictest2556-21ssh23:21:551
331899140,10cyclictest25437-21sh00:12:011
331899140,10cyclictest24773-21sh22:40:031
331899140,10cyclictest21736-21sh22:56:041
331899140,10cyclictest18499-21ssh22:35:461
331899140,10cyclictest16704-21ssh21:21:271
331899140,10cyclictest14720-21rm00:05:301
331899140,10cyclictest12958-21sh21:55:581
331899140,10cyclictest11688-21/usr/sbin/munin20:35:151
331899140,10cyclictest10371-21runrttasks19:23:141
331899140,10cyclictest1033-21sh22:25:481
331499146,7cyclictest28560-21/usr/sbin/munin23:55:170
331499146,6cyclictest28376-21head23:00:140
331499145,7cyclictest24572-21sed23:35:000
331499145,7cyclictest15529-21grep00:25:130
331499143,8cyclictest20327-21vmstat20:50:260
331499142,8cyclictest6551-21entropy19:15:160
331499142,10cyclictest14680-21processes21:20:200
331499141,9cyclictest3267-21latency_hist19:10:010
331499141,9cyclictest17387-21taskset19:35:310
3314991411,2cyclictest5772-21apt-key19:15:000
331499141,10cyclictest7381-21sh23:25:130
331499141,10cyclictest18686-21ssh21:40:460
331499141,10cyclictest14132-21grep20:40:110
331499140,7cyclictest30977-21sh21:30:100
331499140,7cyclictest16678-21/usr/sbin/munin20:45:140
331499140,7cyclictest103-1kworker/0:1H+kblockd19:20:550
331499140,12cyclictest28316-21/usr/sbin/munin22:05:140
331499140,12cyclictest25534-21perl22:40:130
331499140,12cyclictest12491-21sh00:05:030
331499140,12cyclictest10788-21uname23:45:160
331499140,11cyclictest4266-21sh23:05:080
331499140,10cyclictest8072-21sh21:16:340
331499140,10cyclictest7956-21memory20:25:180
331499140,10cyclictest774-21runrttasks21:09:420
331499140,10cyclictest774-21runrttasks20:20:330
331499140,10cyclictest774-21runrttasks20:00:290
331499140,10cyclictest3676-21ssh21:50:360
331499140,10cyclictest30498-21sh21:10:420
331499140,10cyclictest24650-21sh23:35:020
331499140,10cyclictest24056-21ntpq19:50:180
331499140,10cyclictest24052-21sh23:52:540
331499140,10cyclictest21982-21ssh23:15:030
331499140,10cyclictest13349-21runrttasks20:35:290
331499140,10cyclictest12859-21runrttasks19:26:390
331499140,10cyclictest10501-21grep21:55:130
333199135,6cyclictest6532-21df19:15:144
333199135,6cyclictest3697-21/usr/sbin/munin19:10:114
333199135,6cyclictest26362-21irqstats21:05:164
333199135,6cyclictest16338-21cut20:45:114
333199130,9cyclictest32248-21/usr/sbin/munin20:10:144
333199130,9cyclictest29953-21atemtest20:05:104
332799131,9cyclictest7996-21runrttasks00:20:243
332799131,9cyclictest734-21sshd22:50:413
332799131,9cyclictest734-21sshd22:50:413
332799130,9cyclictest9876-21sh23:45:083
332799130,9cyclictest774-21runrttasks21:35:053
332799130,9cyclictest4871-21latency00:00:163
332799130,9cyclictest3578-21/usr/sbin/munin19:10:133
332799130,9cyclictest31382-21sh23:20:143
332799130,9cyclictest27617-21diskmemload21:50:003
332799130,9cyclictest26196-21missed_timers19:55:173
332799130,9cyclictest24761-21sh23:35:073
332799130,9cyclictest24255-21cat21:00:163
332799130,9cyclictest24255-21cat21:00:153
332799130,9cyclictest22037-21sh22:20:033
332799130,9cyclictest2150-21aten2_r5power_c20:15:113
332799130,9cyclictest20827-21tr19:45:123
332799130,9cyclictest20386-21runrttasks20:51:103
332799130,9cyclictest15783-21runrttasks20:41:553
332799130,9cyclictest13771-21df19:30:123
332799130,9cyclictest12158-21uptime21:55:243
332799130,9cyclictest1183-21/usr/sbin/munin23:40:113
332799130,9cyclictest11678-21taskset20:35:023
332799130,9cyclictest11294-21chrt20:33:493
332799130,10cyclictest18695-21cstates19:40:123
332799130,10cyclictest15917-21gltestperf00:25:153
332299131,9cyclictest15202-21runrttasks19:32:072
332299131,9cyclictest1510-21sh21:31:072
332299131,8cyclictest25233-21df_inode23:35:122
332299130,9cyclictest8112-21taskset19:19:072
332299130,9cyclictest774-21runrttasks21:05:162
332299130,9cyclictest774-21runrttasks20:50:022
332299130,9cyclictest774-21runrttasks19:50:152
332299130,9cyclictest711-21snmpd19:41:482
332299130,9cyclictest6512-21taskset00:20:142
332299130,9cyclictest3919-21/usr/sbin/munin00:00:182
332299130,9cyclictest30847-21runrttasks22:25:052
332299130,9cyclictest2760-21grep20:15:152
332299130,9cyclictest27283-21sh23:55:022
332299130,9cyclictest25318-21/usr/sbin/munin19:55:132
332299130,9cyclictest22094-21/usr/sbin/munin20:55:192
332299130,9cyclictest22094-21/usr/sbin/munin20:55:182
332299130,9cyclictest21378-21/usr/sbin/munin00:10:142
332299130,9cyclictest19098-21/usr/sbin/munin22:00:162
332299130,9cyclictest16460-21/usr/sbin/munin00:25:192
332299130,9cyclictest16369-21gltestperf19:35:152
332299130,9cyclictest14851-21ssh21:20:232
332299130,9cyclictest1211-21aten2_r5power_c23:40:112
332299130,10cyclictest8048-21grep20:25:182
332299130,10cyclictest30321-21sh22:05:452
332299130,10cyclictest29578-21chrt20:04:142
332299130,10cyclictest14027-21aten2_r5power_c20:40:122
331899135,6cyclictest9395-21aten2_r5power_c20:30:101
331899135,6cyclictest6488-21cpuspeed_turbos19:15:151
331899135,6cyclictest27791-21cpu20:00:121
331899135,6cyclictest20848-21cpu19:45:121
331899135,6cyclictest13757-21expr19:30:121
331899131,9cyclictest4119-21fschecks_time19:10:121
331899131,9cyclictest22129-21sh22:20:081
331899131,9cyclictest16818-21cat23:30:141
331899130,9cyclictest774-21runrttasks20:55:571
331899130,9cyclictest774-21runrttasks20:55:561
331899130,9cyclictest711-21snmpd21:02:271
331899130,9cyclictest711-21snmpd21:02:271
331899130,9cyclictest4179-21chrt23:05:031
331899130,9cyclictest3227-21open_inodes20:15:181
331899130,9cyclictest27421-21sh23:55:081
331899130,9cyclictest27090-21taskset19:57:001
331899130,9cyclictest23015-21/usr/sbin/munin19:50:131
331899130,9cyclictest16317-21/usr/sbin/munin20:45:201
331899130,9cyclictest14963-21ssh00:25:021
331899130,10cyclictest30275-21tr20:05:131
331899130,10cyclictest26136-21ssh00:30:391
331899130,10cyclictest24737-21ssh23:35:061
331899130,10cyclictest19236-21fschecks_time20:50:151
331899130,10cyclictest13803-21/usr/sbin/munin23:10:151
331499131,9cyclictest1615-21taskset20:12:070
331499130,9cyclictest774-21runrttasks19:55:100
331499130,9cyclictest11012-21unixbench_singl20:30:220
331499130,10cyclictest2492-21df20:15:110
331499130,10cyclictest24264-21needreboot21:00:160
331499130,10cyclictest24264-21needreboot21:00:160
331499130,10cyclictest22448-21sed20:55:230
331499130,10cyclictest22448-21sed20:55:230
331499130,10cyclictest18298-21cmp19:40:010
331499130,10cyclictest13410-21runrttasks19:30:050
332799120,9cyclictest24683-21chrt19:50:373
332299121,8cyclictest281-21jbd2/md0-819:25:312
332299120,9cyclictest29944-21/usr/sbin/munin20:05:132
332299120,9cyclictest21377-21sh19:45:142
332299120,8cyclictest32325-21aten2_r5power_p20:10:142
331899120,9cyclictest15789-21aten2_r5power_p19:35:111
38062110,1sleep50-21swapper/521:50:435
332799110,8cyclictest3534-21latency_hist19:10:013
331899110,7cyclictest334450cat19:10:011
270992110,1sleep50-21swapper/500:13:165
145912110,1sleep50-21swapper/500:05:265
29752100,1sleep50-21swapper/522:27:145
271552100,1sleep50-21swapper/522:22:375
266592100,1sleep50-21swapper/523:17:185
193712100,1sleep50-21swapper/500:27:075
107652100,1sleep50-21swapper/500:04:025
7894290,0sleep50-21swapper/500:39:385
33229990,7cyclictest3334-21latency_hist19:10:012
27340280,1sleep50-21swapper/521:06:285
22942280,0sleep50-21swapper/523:52:065
22539280,1sleep50-21swapper/519:47:465
12972280,1sleep50-21swapper/519:28:135
23834263,1sleep50-21swapper/522:57:375
15375260,1sleep50-21swapper/523:29:565
5386250,3sleep50-21swapper/523:42:045
32247252,1sleep50-21swapper/520:10:135
3129250,1sleep50-21swapper/523:59:555
23342251,1sleep50-21swapper/500:30:015
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional