You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 02:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Sat May 02, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3044321420,0sleep50-21swapper/523:52:585
248332920,0sleep20-21swapper/222:02:392
2074639920,0EcMasterDemoSyn2075629tAtEmLog_022:35:325
1545739910,0EcMasterDemoSyn1546729tAtEmLog_022:15:145
3151539900,0EcMasterDemoSyn3152529tAtEmLog_020:13:325
71039890,0EcMasterDemoSyn72629tAtEmLog_021:49:535
606439840,0EcMasterDemoSyn607429tAtEmLog_023:21:095
2341539840,0EcMasterDemoSyn2342529tAtEmLog_000:06:485
2020439840,0EcMasterDemoSyn2021429tAtEmLog_023:11:005
1030239840,0EcMasterDemoSyn1031129tAtEmLog_021:19:275
996139830,0EcMasterDemoSyn997129tAtEmLog_021:54:565
37439820,0EcMasterDemoSyn38429tAtEmLog_019:07:365
2171339820,0EcMasterDemoSyn2172329tAtEmLog_019:53:155
108139810,0EcMasterDemoSyn109129tAtEmLog_023:36:225
9102760,0sleep0151ktimers/023:00:240
266792240,0sleep20-21swapper/221:28:072
1517439210,0EcMasterDemoSyn15180-21ps23:26:140
65099181,13cyclictest16-1pr/legacy23:16:041
64799185,9cyclictest2327-21sh21:15:020
64799180,14cyclictest3519-21sh00:14:170
64799180,10cyclictest29412-21sh23:52:160
1831029180,8tAtEmLog_026123-21chrt00:26:125
1831029180,8tAtEmLog_026123-21chrt00:26:115
990899170,10tDemoTimingTask10391-21taskset19:30:005
858299170,10tDemoTimingTask8982-21taskset20:35:005
65799179,6cyclictest25164-21grep23:50:124
65399174,9cyclictest3513-21latency21:15:173
65399170,9cyclictest21734-21rm23:30:133
65399170,9cyclictest17583-21ssh23:27:553
64799175,9cyclictest8820-21sh22:11:240
64799175,9cyclictest32461-21diskstats00:30:120
64799170,9cyclictest780-21runrttasks19:33:470
64799170,9cyclictest31225-21sh22:05:440
64799170,9cyclictest16583-21sh00:39:400
64799170,9cyclictest14564-21sh23:44:310
64799170,14cyclictest23165-21sh23:49:150
64799170,14cyclictest14081-21ssh21:39:160
64799170,14cyclictest12674-21sh00:00:340
2518329170,10tAtEmLog_025582-21taskset21:10:015
1518599170,10tDemoTimingTask20843-21chrt23:30:015
657991612,3cyclictest13783-21fschecks_count22:50:134
65799160,10cyclictest8111-21sh21:53:394
65399169,5cyclictest9781-21tr20:35:153
65399169,5cyclictest9781-21tr20:35:153
653991612,3cyclictest8703-21cron20:35:013
65199163,9cyclictest20541-21vmstat22:35:232
65099164,8cyclictest28568-21awk20:05:241
650991610,5cyclictest10826-21grep19:30:131
65099160,9cyclictest15353-21sh21:21:351
64799160,8cyclictest21452-21/usr/sbin/munin21:25:150
64799160,14cyclictest9853-21grep22:30:120
64799160,14cyclictest9853-21grep22:30:110
64799160,13cyclictest780-21runrttasks19:52:330
2518499160,9tDemoTimingTask25188-21chrt21:09:195
1946299160,10tDemoTimingTask28522-21chrt22:05:005
65799150,9cyclictest27458-21sh22:21:314
65399157,6cyclictest14350-21grep20:45:153
65399154,6cyclictest1200-21grep19:10:143
65399150,8cyclictest25281-21diskmemload22:25:203
65399150,11cyclictest30518-21ssh21:48:033
65199153,9cyclictest10434-21cat20:35:222
65199153,9cyclictest10434-21cat20:35:212
65199151,10cyclictest7050-21runrttasks19:20:202
65199151,10cyclictest4594-21ssh21:34:012
65199151,10cyclictest31678-21ssh22:24:312
65199150,9cyclictest13606-21rm21:38:572
65099155,8cyclictest18510-21/usr/sbin/munin20:55:121
65099151,10cyclictest16473-21sh00:39:351
65099151,10cyclictest14501-21runrttasks19:36:341
65099150,9cyclictest28844-21diskmemload22:22:311
65099150,10cyclictest32573-21ssh22:06:421
65099150,10cyclictest26400-21ssh22:03:431
64799157,6cyclictest29914-21tr20:10:120
64799156,7cyclictest24095-21grep22:20:130
64799156,7cyclictest1566-21ssh21:31:520
647991511,3cyclictest1-21systemd22:40:000
64799151,11cyclictest16-1pr/legacy00:06:470
64799150,8cyclictest3188-21sh22:25:530
64799150,7cyclictest25289-21awk21:10:000
64799150,6cyclictest27386-21ps00:27:050
64799150,6cyclictest27386-21ps00:27:050
64799150,13cyclictest174-1kworker/0:1H+kblockd23:10:180
64799150,12cyclictest780-21runrttasks20:17:220
64799150,11cyclictest5496-21sh21:51:500
64799150,11cyclictest30060-21ssh23:35:060
64799150,11cyclictest21859-21apt-get22:55:030
404699150,9tDemoTimingTask31524-21sleep00:34:565
65799145,7cyclictest8790-21/usr/sbin/munin19:25:164
65799141,9cyclictest955-21runrttasks19:10:024
65799141,9cyclictest27312-21diskmemload21:10:214
65799140,9cyclictest22010-21rm22:00:404
65799140,8cyclictest17743-21ssh21:59:024
65799140,11cyclictest326-21systemd-journal20:40:004
65799140,11cyclictest326-21systemd-journal20:40:004
65799140,10cyclictest780-21runrttasks20:08:374
65799140,10cyclictest3568-21sh22:26:094
65799140,10cyclictest3561-21ssh23:38:064
65799140,10cyclictest3262-21cat20:20:234
65799140,10cyclictest28426-21cut22:40:144
65799140,10cyclictest23389-21sh22:37:234
65799140,10cyclictest2168-21ssh00:30:504
65799140,10cyclictest15655-21ssh23:45:044
65799140,10cyclictest14994-21taskset21:21:214
65799140,10cyclictest14516-21sh23:25:464
65799140,10cyclictest1403-21sh22:07:334
65799140,10cyclictest12898-21sh22:31:194
65799140,10cyclictest12898-21sh22:31:194
65799140,10cyclictest12698-21ssh22:49:554
65399147,6cyclictest7394-21grep23:40:143
65399143,8cyclictest728-21lldpd21:03:043
65399143,8cyclictest25313-21wc22:55:493
65399141,9cyclictest6592-21/usr/sbin/munin21:35:173
65399141,9cyclictest31972-21/usr/sbin/munin20:15:143
65399141,9cyclictest22871-21diskmemload23:49:023
65399141,9cyclictest16955-21sh00:03:333
65399141,9cyclictest11880-21ssh22:13:353
65399141,10cyclictest27860-21sh22:21:483
65399140,9cyclictest1353-21sh21:50:043
65399140,12cyclictest7425-21ssh23:58:163
65399140,10cyclictest4214-21ssh22:45:053
65399140,10cyclictest3496-21ssh23:20:133
65399140,10cyclictest32397-21ssh00:11:483
65399140,10cyclictest31908-21ssh00:30:063
65399140,10cyclictest30228-21sensors_fan22:05:193
65399140,10cyclictest29741-21ssh21:30:033
65399140,10cyclictest27744-21sh23:51:063
65399140,10cyclictest24772-21ssh21:26:503
65399140,10cyclictest24563-21/usr/sbin/munin20:00:233
65399140,10cyclictest24314-21sh22:02:183
65399140,10cyclictest23969-21runrttasks23:13:403
65399140,10cyclictest23052-21/usr/sbin/munin00:25:193
65399140,10cyclictest23052-21/usr/sbin/munin00:25:183
65399140,10cyclictest22562-21sh00:06:123
65399140,10cyclictest2240-21ssh23:37:093
65399140,10cyclictest22329-21chrt21:43:413
65399140,10cyclictest21295-21ssh22:35:533
65399140,10cyclictest20884-21ssh22:54:193
65399140,10cyclictest17265-21rm22:15:453
65399140,10cyclictest16023-21sed00:20:253
65399140,10cyclictest15449-21ssh21:21:413
65399140,10cyclictest14873-21ssh21:57:033
65399140,10cyclictest11545-21ssh23:06:133
65399140,10cyclictest10403-21/usr/sbin/munin19:30:173
65199145,7cyclictest468-21ssh21:49:452
65199144,8cyclictest896-21/usr/sbin/munin22:25:132
65199141,9cyclictest6038-21diskmemload21:16:302
65199141,9cyclictest25787-21/usr/sbin/munin21:10:112
65199141,9cyclictest22190-21/usr/sbin/munin19:55:132
65199141,9cyclictest21931-21proc_pri21:00:192
65199141,9cyclictest16805-21sh22:15:262
65199141,10cyclictest28001-21ssh23:33:502
65199140,9cyclictest780-21runrttasks20:20:292
65199140,9cyclictest3610-21diskmemload00:31:502
65199140,9cyclictest25736-21sh22:56:082
65199140,11cyclictest19721-21rm23:10:422
65199140,11cyclictest16086-21/usr/sbin/munin20:50:172
65199140,10cyclictest8310-21runrttasks20:30:252
65199140,10cyclictest7814-21ssh22:10:432
65199140,10cyclictest780-21runrttasks22:51:382
65199140,10cyclictest7575-21rm22:46:172
65199140,10cyclictest5400-21sh21:51:452
65199140,10cyclictest326-21systemd-journal20:29:552
65199140,10cyclictest3215-21ssh22:44:342
65199140,10cyclictest31813-21ssh22:06:092
65199140,10cyclictest26532-21cat23:50:232
65199140,10cyclictest24240-21latency_hist21:44:592
65199140,10cyclictest2357-21sh00:13:272
65199140,10cyclictest18897-21ssh23:28:502
65199140,10cyclictest18012-21tune2fs19:45:212
65199140,10cyclictest18003-21rm22:34:572
65199140,10cyclictest18003-21rm22:34:562
65199140,10cyclictest17877-21ssh23:45:302
65199140,10cyclictest17685-21sh21:23:132
65199140,10cyclictest14301-21cut20:45:152
65199140,10cyclictest14041-21ssh00:20:092
65199140,10cyclictest12957-21sh00:37:072
65199140,10cyclictest12366-21unixbench-2d00:00:262
65199140,10cyclictest11513-21ssh23:06:122
65099144,6cyclictest6303-21expr19:20:121
65099143,8cyclictest17855-21df19:45:201
65099142,9cyclictest3085-21ssh23:37:471
65099141,9cyclictest8724-21sh21:35:311
65099141,9cyclictest857-21ssh00:12:231
65099141,9cyclictest5260-21sh22:10:041
65099141,9cyclictest1947-21/usr/sbin/munin20:20:181
65099141,9cyclictest15755-21sh23:45:091
65099141,10cyclictest28029-21ssh21:10:351
65099141,10cyclictest2197-21hddtemp_smartct23:55:151
65099141,10cyclictest21733-21runrttasks22:18:551
65099141,10cyclictest16-1pr/legacy22:25:211
65099140,9cyclictest23168-21/usr/sbin/munin21:05:161
65099140,12cyclictest23989-21ssh23:31:041
65099140,12cyclictest19359-21ssh23:10:261
65099140,10cyclictest9554-21/usr/sbin/munin22:30:161
65099140,10cyclictest9554-21/usr/sbin/munin22:30:151
65099140,10cyclictest780-21runrttasks00:25:581
65099140,10cyclictest780-21runrttasks00:25:581
65099140,10cyclictest7767-21proc_pri20:30:181
65099140,10cyclictest7607-21sh21:17:361
65099140,10cyclictest7203-21ssh23:21:571
65099140,10cyclictest6783-21/usr/sbin/munin23:40:181
65099140,10cyclictest5085-21taskset21:34:211
65099140,10cyclictest3288-21ssh22:44:381
65099140,10cyclictest28710-21sh22:58:141
65099140,10cyclictest27597-21runrttasks23:51:001
65099140,10cyclictest25688-21sh21:45:151
65099140,10cyclictest24915-21ssh22:38:271
65099140,10cyclictest24508-21sh21:26:381
65099140,10cyclictest19042-21ssh21:59:581
65099140,10cyclictest18558-21ssh22:52:401
65099140,10cyclictest18033-21ssh21:40:401
65099140,10cyclictest17144-21ssh00:21:131
65099140,10cyclictest1350-21ssh21:50:041
65099140,10cyclictest13460-21ssh00:01:081
65099140,10cyclictest11002-21ssh23:05:521
64799146,6cyclictest3954-21uname23:20:180
64799146,6cyclictest12686-21memory21:20:160
64799145,7cyclictest24357-21wc23:31:190
64799145,7cyclictest15595-21ssh23:09:060
64799143,8cyclictest4914-21grep20:25:150
64799143,6cyclictest26459-21gltestperf21:10:130
64799142,9cyclictest780-21runrttasks20:58:350
64799141,9cyclictest14042-21sh00:20:080
64799141,7cyclictest11065-21latency_hist20:40:010
64799141,7cyclictest11065-21latency_hist20:40:000
64799141,11cyclictest13714-21cat20:45:000
64799141,10cyclictest16-1pr/legacy19:48:100
64799140,9cyclictest14995-21taskset20:45:220
64799140,7cyclictest9641-21ssh22:47:450
64799140,7cyclictest882-21cron23:55:000
64799140,12cyclictest28591-21sh21:46:410
64799140,12cyclictest27827-21which23:15:200
64799140,12cyclictest12416-21grep21:55:250
64799140,10cyclictest9250-21runrttasks00:16:590
64799140,10cyclictest780-21runrttasks20:50:390
64799140,10cyclictest780-21runrttasks19:37:260
64799140,10cyclictest28726-21ssh22:40:170
64799140,10cyclictest25004-21sh22:02:460
64799140,10cyclictest16619-21diskmemload22:51:170
64799140,10cyclictest15161-21ssh22:15:120
1931699140,8tDemoTimingTask19427-21taskset23:46:345
148599140,8tDemoTimingTask1516-21taskset20:18:395
65799135,6cyclictest10226-21find00:35:214
65799133,8cyclictest13759-21/usr/sbin/munin20:45:124
65799131,9cyclictest30701-21runrttasks20:10:184
65799131,8cyclictest25638-21if_enp6s021:45:164
65799130,9cyclictest8589-21ssh00:16:324
65799130,9cyclictest780-21runrttasks23:05:034
65799130,9cyclictest780-21runrttasks20:29:074
65799130,9cyclictest716-21snmpd19:16:014
65799130,9cyclictest6769-21wget20:30:124
65799130,9cyclictest5691-21/usr/sbin/munin19:20:134
65799130,9cyclictest5275-21ssh22:10:054
65799130,9cyclictest2829-21sh21:32:464
65799130,9cyclictest28260-21sh00:10:024
65799130,9cyclictest2650-21munin-node23:40:114
65799130,9cyclictest24624-21wget20:00:124
65799130,9cyclictest23922-21irqstats21:05:164
65799130,9cyclictest23015-21ssh00:25:084
65799130,9cyclictest23015-21ssh00:25:074
65799130,9cyclictest22693-21sh22:55:114
65799130,9cyclictest22356-21awk19:55:124
65799130,9cyclictest21092-21python319:50:224
65799130,9cyclictest21076-21/usr/sbin/munin23:30:134
65799130,9cyclictest20722-21ssh21:25:074
65799130,9cyclictest19573-21sh00:05:144
65799130,9cyclictest19159-21runrttasks19:45:374
65799130,9cyclictest17244-21sh23:10:034
65799130,9cyclictest16248-21perl21:40:134
65799130,9cyclictest16078-21taskset20:50:024
65799130,9cyclictest15132-21/usr/sbin/munin19:40:114
65799130,9cyclictest14740-21/usr/sbin/munin22:15:164
65799130,9cyclictest1440-21runrttasks20:18:164
65799130,9cyclictest13808-21ntpq19:35:174
65799130,9cyclictest12700-21sh00:00:344
65799130,9cyclictest11390-21/usr/sbin/munin20:40:174
65799130,9cyclictest10936-21cut19:30:134
65799130,7cyclictest6589-21sh21:35:094
65799130,10cyclictest5489-21sh23:20:444
65799130,10cyclictest5239-21ssh21:15:564
65799130,10cyclictest28881-21ssh23:15:474
65799130,10cyclictest2799-21ssh23:01:264
65799130,10cyclictest2567-21grep23:55:184
65799130,10cyclictest14870-21sed00:20:164
65399139,3cyclictest26798-21cpuspeed_turbos23:15:143
65399139,3cyclictest20056-21cpuspeed_turbos19:50:143
65399134,7cyclictest15443-21grep19:40:113
65399131,8cyclictest11527-21diskmemload22:30:253
65399131,8cyclictest11527-21diskmemload22:30:243
65399130,9cyclictest8261-21cpuspeed_turbos19:25:113
65399130,9cyclictest5973-21sh00:15:183
65399130,9cyclictest4976-21taskset19:15:263
65399130,9cyclictest29644-21cut20:10:113
65399130,9cyclictest27506-21sh22:40:013
65399130,9cyclictest18449-21apt-get20:55:113
65399130,9cyclictest16086-21/usr/sbin/munin20:50:233
65399130,9cyclictest12758-21/usr/sbin/munin19:35:133
65399130,9cyclictest11102-21rm00:35:463
65399130,10cyclictest780-21runrttasks19:55:273
65399130,10cyclictest6932-21mailstats19:20:163
65399130,10cyclictest4649-21df20:25:123
65399130,10cyclictest2986-21nfs4_client20:20:173
65399130,10cyclictest28592-21sed20:05:253
65399130,10cyclictest26628-21if_enp6s021:10:153
65399130,10cyclictest24763-21unixbench_singl21:05:243
65399130,10cyclictest1840-21ssh23:00:483
65399130,10cyclictest18291-21latency19:45:243
65399130,10cyclictest12625-21cut20:40:253
65199135,6cyclictest8571-21tune2fs19:25:132
65199135,6cyclictest30632-21expr23:35:142
65199131,9cyclictest5090-21/usr/sbin/munin00:15:132
65199131,9cyclictest14346-21/usr/sbin/munin19:35:252
65199131,7cyclictest1349-21fschecks_count19:10:152
65199130,9cyclictest8227-21ssh23:40:202
65199130,9cyclictest780-21runrttasks19:50:442
65199130,9cyclictest780-21runrttasks19:33:542
65199130,9cyclictest31972-21/usr/sbin/munin20:15:122
65199130,9cyclictest31597-21ssh23:00:042
65199130,9cyclictest26969-21/usr/sbin/munin20:05:232
65199130,9cyclictest26849-21ssh23:15:142
65199130,9cyclictest25508-21sed20:00:172
65199130,9cyclictest24369-21processes00:25:192
65199130,9cyclictest24369-21processes00:25:182
65199130,9cyclictest23168-21/usr/sbin/munin21:05:182
65199130,9cyclictest19650-21/usr/sbin/munin00:05:262
65199130,9cyclictest18816-21forks20:55:132
65199130,9cyclictest15721-21gltestperf19:40:132
65199130,9cyclictest1291-21sh23:55:052
65199130,8cyclictest3297-21cpuspeed23:20:132
65199130,10cyclictest3821-21grep19:15:132
65199130,10cyclictest29943-21tr20:10:122
65199130,10cyclictest13139-21runrttasks20:41:442
65199130,10cyclictest11207-21fschecks_time21:55:142
65099130,9cyclictest780-21runrttasks21:00:021
65099130,9cyclictest780-21runrttasks20:39:181
65099130,9cyclictest780-21runrttasks20:39:171
65099130,9cyclictest780-21runrttasks20:25:531
65099130,9cyclictest716-21snmpd19:57:091
65099130,9cyclictest4329-21/usr/sbin/munin22:45:191
65099130,9cyclictest2072-21proc_pri19:10:201
65099130,9cyclictest20428-21sh00:05:211
65099130,9cyclictest20044-21cpuspeed_turbos19:50:121
65099130,9cyclictest15513-21runrttasks20:47:011
65099130,9cyclictest15132-21/usr/sbin/munin19:40:101
65099130,9cyclictest14115-21sh23:25:301
65099130,10cyclictest8250-21cpuspeed_turbos19:25:111
65099130,10cyclictest5901-21basename00:15:181
65099130,10cyclictest4193-21memory19:15:161
65099130,10cyclictest31822-21ssh23:00:131
65099130,10cyclictest29971-21df20:10:121
65099130,10cyclictest25169-21fschecks_time20:00:161
65099130,10cyclictest2077-21ssh00:30:441
65099130,10cyclictest17769-21runrttasks20:50:441
65099130,10cyclictest1223-21vmstat20:15:251
65099130,10cyclictest12117-21cat20:40:211
64799132,8cyclictest8174-21cpu19:25:110
64799131,9cyclictest16-1pr/legacy19:58:180
64799131,10cyclictest16-1pr/legacy20:08:260
64799130,9cyclictest780-21runrttasks21:40:110
64799130,9cyclictest780-21runrttasks20:20:220
64799130,9cyclictest26466-21taskset20:03:020
64799130,7cyclictest174-1kworker/0:1H+kblockd21:02:220
64799130,7cyclictest174-1kworker/0:1H+kblockd19:42:020
64799130,7cyclictest174-1kworker/0:1H+kblockd19:20:530
64799130,10cyclictest780-21runrttasks19:10:440
64799130,10cyclictest3589-21tr19:15:120
622229130,11tAtEmLog_06326-21chrt20:29:475
2993839130,1EcMasterDemoSyn671rcuc/522:40:355
2844899131,10tDemoTimingTask4194-21taskset23:56:005
1433729130,2tAtEmLog_018438-21chrt00:04:375
1313029130,2tAtEmLog_0138532sleep500:37:445
65799120,8cyclictest20882-21/usr/sbin/munin21:00:124
65799120,7cyclictest18444-21/usr/sbin/munin20:55:114
65199121,6cyclictest740-21latency_hist19:10:002
64799120,9cyclictest6821-21awk20:30:110
64799120,10cyclictest609-21latency_hist19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional