You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-19 - 08:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4.osadl.org (updated Sun May 19, 2024 00:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1186484992824,2cyclictest1480015-21kworker/u8:0-rpciod00:19:592
1186482992723,2cyclictest1239932-21turbostat.cron20:05:001
1186490992522,2cyclictest1458879-21ssh22:49:593
1186484992421,2cyclictest1323882-21cron21:20:002
1186478992419,3cyclictest260-21systemd-journal22:45:000
1186482992317,3cyclictest0-21swapper/122:19:591
1186478992218,2cyclictest1346385-21turbostat.cron21:35:000
1186490992118,2cyclictest2955-21cron22:30:003
118648299211,11cyclictest1503788-21munin-run23:20:001
1186484992017,2cyclictest514-21in:imuxsock22:40:002
1186484992017,2cyclictest1572273-21gltestperf00:05:162
1186482992017,2cyclictest1186471-21cyclictest19:50:271
1186482992016,3cyclictest1354978-21gltestperf21:40:161
1186482992016,2cyclictest0-21swapper/122:05:181
1186490991916,2cyclictest1519414-21gltestperf23:30:163
1186490991916,2cyclictest1276659-21unixbench_singl20:35:343
1186490991915,3cyclictest1613451-21ntp_kernel_pll_00:30:243
1186484991917,1cyclictest1306362-21kthreadcore21:05:212
1186484991916,2cyclictest0-21swapper/220:05:002
1186482991916,2cyclictest1527078-21irqcore23:35:181
1186478991914,2cyclictest60650irq/128-enp0s29f1:rx-000:01:070
1186478991913,2cyclictest60650irq/128-enp0s29f1:rx-023:35:010
1186490991816,1cyclictest1209196-21irqcore19:35:183
1186490991815,2cyclictest0-21swapper/323:00:003
1186490991814,2cyclictest0-21swapper/321:20:213
1186484991815,2cyclictest1338850-21idleruntime-cro21:30:012
1186484991815,2cyclictest0-21swapper/200:30:012
1186484991814,2cyclictest0-21swapper/222:50:242
1186484991813,4cyclictest1249935-21unixbench_multi20:10:242
1186482991815,2cyclictest1541169-21latency_hist23:45:011
1186482991814,3cyclictest1343457-21needreboot21:30:251
1186490991714,2cyclictest1351437-21unixbench_singl21:35:323
1186490991714,2cyclictest1256384-21irqcore20:20:163
118648499173,2cyclictest191rcu_preempt23:45:352
1186484991714,2cyclictest1353843-21systemd-run21:40:002
1186484991714,2cyclictest1288587-21gltestperf20:50:162
1186484991714,2cyclictest0-21swapper/200:40:012
1186484991711,4cyclictest0-21swapper/223:55:002
1186482991713,2cyclictest1586475-21latency_hist00:15:001
1186482991712,3cyclictest1219433-21fschecks_time19:45:161
118648299170,3cyclictest1308947-21munin-run21:10:001
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-023:53:060
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-023:10:210
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-022:48:020
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-022:36:350
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-022:26:420
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-022:05:280
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-021:55:330
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-021:52:110
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-021:41:130
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-021:35:360
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-000:28:300
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-000:17:520
1186478991714,2cyclictest60650irq/128-enp0s29f1:rx-000:14:580
1186478991714,2cyclictest0-21swapper/021:10:260
1186478991714,2cyclictest0-21swapper/021:00:010
1186478991713,3cyclictest1538212-21needreboot23:40:230
118649099162,2cyclictest191rcu_preempt23:05:233
1186490991614,1cyclictest1249053-21kthreadcore20:10:183
1186490991614,1cyclictest1239910-21cron20:04:593
1186490991613,2cyclictest1564569-21tr00:00:173
1186490991613,2cyclictest1526882-21gltestperf23:35:163
1186490991613,2cyclictest1449026-21unixbench_multi22:40:333
1186490991613,2cyclictest1208432-21sh19:34:593
1186490991612,3cyclictest0-21swapper/320:55:013
1186490991612,2cyclictest0-21swapper/321:30:013
1186490991611,4cyclictest491ktimers/300:05:503
118648499163,2cyclictest191rcu_preempt23:55:122
1186484991613,2cyclictest1346379-21turbostat.cron21:35:002
1186484991612,3cyclictest1261337-21fschecks_count20:25:152
118648499160,3cyclictest1541348-21sort23:45:012
118648499160,3cyclictest1218930-21cron19:44:592
118648499160,3cyclictest0-21swapper/222:30:002
118648299169,5cyclictest1477059-21tr23:00:181
1186482991613,2cyclictest516-21rs:main1
1186482991613,2cyclictest1518725-21idleruntime-cro23:30:001
1186482991613,2cyclictest1518725-21idleruntime-cro23:30:001
1186482991613,2cyclictest1423026-21sort22:25:221
1186482991612,3cyclictest3150ktimers/122:10:121
1186482991612,3cyclictest1354023-21apt-config21:40:001
1186482991612,3cyclictest0-21swapper/100:40:011
1186482991612,2cyclictest1282302-21idleruntime-cro20:45:011
1186482991612,2cyclictest0-21swapper/121:55:141
1186482991611,3cyclictest0-21swapper/100:05:141
1186478991614,2cyclictest60650irq/128-enp0s29f1:rx-019:35:490
1186478991614,1cyclictest60650irq/128-enp0s29f1:rx-023:20:030
1186478991614,1cyclictest60650irq/128-enp0s29f1:rx-022:22:290
1186478991614,1cyclictest60650irq/128-enp0s29f1:rx-020:20:280
1186478991614,1cyclictest60650irq/128-enp0s29f1:rx-019:51:260
1186478991614,1cyclictest60650irq/128-enp0s29f1:rx-000:36:070
1186478991613,2cyclictest61450irq/129-enp0s29f1:tx-020:15:140
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-023:55:160
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-023:45:170
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-023:31:240
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-023:15:440
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-023:09:500
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-023:04:070
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-022:56:110
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-022:50:470
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-022:15:170
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-022:00:260
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-021:45:200
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-021:26:190
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-021:18:330
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-020:30:260
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-020:30:250
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-019:45:300
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-019:15:180
1186478991613,2cyclictest60650irq/128-enp0s29f1:rx-000:20:260
1186478991613,2cyclictest1399619-21fschecks_count22:10:170
1186478991613,2cyclictest0-21swapper/019:25:000
1186478991612,3cyclictest1613862-21grep00:30:280
1186478991610,5cyclictest1572461-21sed00:05:180
1186490991513,1cyclictest1594793-21sed00:20:173
1186490991512,2cyclictest1544460-21kthreadcore23:45:243
1186490991512,2cyclictest1384192-21latency_hist22:00:013
1186490991512,2cyclictest1354667-21cpuspeed_turbos21:40:133
1186490991512,2cyclictest1340374-21kthreadcore21:30:223
1186490991512,2cyclictest1203153-21idleruntime-cro19:30:003
1186490991511,3cyclictest1196325-21needreboot19:15:233
1186490991511,2cyclictest0-21swapper/300:25:163
118648499154,1cyclictest1308945-21diskmemload22:12:572
118648499153,1cyclictest1308945-21diskmemload22:04:242
118648499151,2cyclictest191rcu_preempt23:32:332
1186484991512,2cyclictest401ktimers/222:15:212
1186484991512,2cyclictest1250983-21gltestperf20:15:162
118648499151,1cyclictest1308945-21diskmemload00:30:122
1186484991511,3cyclictest1512466-21irqcore23:25:182
1186484991511,3cyclictest1512466-21irqcore23:25:172
1186484991510,3cyclictest260-21systemd-journal22:22:222
1186484991510,3cyclictest260-21systemd-journal19:17:222
118648499150,1cyclictest0-21swapper/220:20:092
118648299158,5cyclictest1332292-21cut21:25:181
1186482991512,2cyclictest1240272-21cpuspeed_turbos20:05:151
1186482991511,3cyclictest1602003-21gltestperf00:25:141
1186482991511,2cyclictest1566937-21kthreadcore00:00:241
1186482991511,2cyclictest0-21swapper/100:30:281
118647899158,5cyclictest1192005-21awk19:15:000
1186478991513,2cyclictest61450irq/129-enp0s29f1:tx-020:45:100
1186478991513,2cyclictest60650irq/128-enp0s29f1:rx-023:25:270
1186478991513,2cyclictest60650irq/128-enp0s29f1:rx-023:25:270
1186478991513,2cyclictest60650irq/128-enp0s29f1:rx-020:54:500
1186478991512,2cyclictest60650irq/128-enp0s29f1:rx-022:30:500
1186478991512,2cyclictest60650irq/128-enp0s29f1:rx-021:20:170
1186478991512,2cyclictest60650irq/128-enp0s29f1:rx-021:05:360
1186478991512,2cyclictest60650irq/128-enp0s29f1:rx-020:40:320
1186478991512,2cyclictest60650irq/128-enp0s29f1:rx-020:10:130
1186478991512,2cyclictest60650irq/128-enp0s29f1:rx-019:32:230
118649099148,4cyclictest1584090-21users00:10:263
118649099148,4cyclictest1553558-21memory23:50:263
118649099145,7cyclictest1589287-21kthreadcore00:15:203
1186490991412,1cyclictest1512006-21gltestperf23:25:153
1186490991412,1cyclictest1512006-21gltestperf23:25:143
1186490991411,2cyclictest1506606-21kthreadcore23:20:193
1186490991411,2cyclictest1489195-21/usr/sbin/munin23:10:253
1186490991411,2cyclictest1419021-21timerwakeupswit22:20:373
1186490991411,2cyclictest1309836-21irqcore21:10:183
1186490991411,2cyclictest1232561-21kthreadcore19:55:223
1186490991411,2cyclictest1224755-21gltestperf19:50:143
1186490991411,2cyclictest1219276-21cpuspeed_turbos19:45:133
1186490991411,2cyclictest0-21swapper/322:00:233
1186490991410,3cyclictest1316829-21cpuspeed_turbos21:15:123
1186490991410,2cyclictest0-21swapper/322:15:003
1186490991410,2cyclictest0-21swapper/321:45:183
118649099140,5cyclictest1371962-21kthreadcore21:50:223
118648499149,4cyclictest1456487-21unixbench_multi22:45:352
118648499143,1cyclictest1308945-21diskmemload21:54:072
1186484991411,2cyclictest1594579-21gltestperf00:20:152
1186484991411,2cyclictest1474051-21latency_hist23:00:012
1186484991411,2cyclictest1429746-21gltestperf22:30:212
1186484991411,2cyclictest1354652-21cpuspeed_turbos21:40:132
1186484991411,2cyclictest1283326-21gltestperf20:45:122
1186484991411,2cyclictest0-21swapper/223:00:172
1186484991411,2cyclictest0-21swapper/200:10:162
1186484991410,2cyclictest1213693-21munin-run19:39:592
1186484991410,2cyclictest0-21swapper/220:40:212
118648499140,2cyclictest191rcu_preempt23:16:112
118648499140,2cyclictest191rcu_preempt23:06:022
118648499140,2cyclictest191rcu_preempt21:15:002
118648299149,4cyclictest311ktimers/121:54:571
118648299149,3cyclictest1261296-21tr20:25:141
118648299149,3cyclictest0-21swapper/122:45:131
118648299148,5cyclictest1328701-21sensors_temp21:20:301
118648299148,4cyclictest1526206-21systemd-run23:34:591
118648299144,2cyclictest833-21runrttasks21:19:561
118648299141,4cyclictest1384393-21apt-get22:00:121
1186482991411,2cyclictest1429546-21cpuspeed_turbos22:30:161
1186482991411,2cyclictest1400619-21kworker/u8:0+xprtiod22:23:581
1186482991411,2cyclictest0-21swapper/123:55:011
1186482991411,2cyclictest0-21swapper/122:54:591
1186482991410,3cyclictest1467143-21head22:55:171
1186482991410,3cyclictest1199242-21kthreadcore19:25:141
118648299140,4cyclictest1298417-21awk21:00:011
1186478991411,2cyclictest60650irq/128-enp0s29f1:rx-019:55:180
1186478991411,2cyclictest60650irq/128-enp0s29f1:rx-019:25:110
1186478991411,2cyclictest260-21systemd-journal21:05:000
1186478991411,2cyclictest1240308-21cpuspeed_turbos20:05:150
1186478991411,2cyclictest1235071-21cpuspeed_turbos20:00:130
118649099139,3cyclictest0-21swapper/321:05:003
118649099139,2cyclictest0-21swapper/322:35:003
118649099138,3cyclictest260-21systemd-journal23:44:593
118649099137,4cyclictest523-21gdbus22:40:003
1186490991311,1cyclictest0-21swapper/320:30:013
1186490991310,2cyclictest0-21swapper/321:00:003
1186490991310,2cyclictest0-21swapper/320:35:003
1186490991310,2cyclictest0-21swapper/320:35:003
118649099130,3cyclictest0-21swapper/300:00:003
118649099130,2cyclictest191rcu_preempt00:35:523
118648499131,2cyclictest191rcu_preempt22:44:492
118648499131,2cyclictest191rcu_preempt21:21:132
1186484991311,1cyclictest0-21swapper/219:10:012
1186484991310,2cyclictest1392242-21gltestperf22:05:162
1186484991310,2cyclictest1377269-21gltestperf21:55:192
1186484991310,2cyclictest1240257-21cpuspeed_turbos20:05:152
1186484991310,2cyclictest0-21swapper/223:10:162
118648499130,2cyclictest191rcu_preempt23:36:442
118648499130,11cyclictest1208424-21awk19:34:592
118648299139,3cyclictest1480015-21kworker/u8:0+xprtiod23:14:291
118648299139,3cyclictest1438224-21kthreadcore22:35:271
118648299139,2cyclictest1289310-21tr20:50:191
118648299139,2cyclictest0-21swapper/119:55:111
118648299139,2cyclictest0-21swapper/119:10:201
118648299137,4cyclictest1597147-21cut00:20:191
1186482991310,2cyclictest311ktimers/121:45:251
1186482991310,2cyclictest2087815-21kworker/u8:1+xprtiod21:12:451
118648299130,11cyclictest1271401-21latency_hist20:35:011
118648299130,11cyclictest1271401-21latency_hist20:35:001
1186478991311,1cyclictest60650irq/128-enp0s29f1:rx-020:36:030
1186478991311,1cyclictest60650irq/128-enp0s29f1:rx-020:25:180
118649099129,2cyclictest1474550-21gltestperf23:00:133
118649099128,3cyclictest1307211-21sshd21:05:233
118649099128,3cyclictest1244762-21unixbench_singl20:05:343
118649099128,2cyclictest0-21swapper/320:45:003
118649099124,1cyclictest191rcu_preempt22:05:123
118649099121,2cyclictest191rcu_preempt22:15:113
1186490991210,1cyclictest0-21swapper/319:10:023
118648499129,2cyclictest1526209-21kworker/u8:2+xprtiod00:01:082
118648499127,3cyclictest1298091-21sshd20:56:232
118648499122,2cyclictest191rcu_preempt21:00:122
1186484991210,1cyclictest1187333-21gltestperf19:10:182
118648299129,2cyclictest509-21systemd-logind19:45:001
118648299129,2cyclictest1504669-21sed23:20:161
118648299129,2cyclictest1250405-21sh20:15:001
118648299129,2cyclictest1192345-21cpuspeed_turbos19:15:141
118648299129,1cyclictest0-21swapper/122:40:241
118648299128,3cyclictest311ktimers/123:09:191
118648299126,4cyclictest1588368-21cut00:15:191
118648299126,4cyclictest1186485-21cat19:10:011
118647899129,2cyclictest60650irq/128-enp0s29f1:rx-019:41:010
118649099119,1cyclictest1497341-21irqcore23:15:163
118649099118,2cyclictest0-21swapper/319:40:143
118649099117,3cyclictest0-21swapper/319:10:013
118649099116,4cyclictest1250926-21perl20:15:143
118648499119,1cyclictest0-21swapper/221:45:242
118648499119,1cyclictest0-21swapper/219:57:272
118648499119,1cyclictest0-21swapper/219:50:142
118648499117,3cyclictest1219276-21cpuspeed_turbos19:45:132
118648499116,4cyclictest497-21dbus-daemon19:30:002
118648499116,4cyclictest1275884-21tr20:35:242
118648499115,4cyclictest1266488-21cut20:30:112
118648499115,4cyclictest1266488-21cut20:30:112
118648299117,3cyclictest1480015-21kworker/u8:0+xprtiod23:58:361
118648299116,3cyclictest1197882-21cron19:25:001
118648299115,4cyclictest1298869-21cut21:00:151
118648299110,8cyclictest1288088-21cat20:50:001
118649099108,1cyclictest3005724-21kworker/3:2+events_power_efficient19:20:053
118648299106,2cyclictest0-21swapper/119:34:591
118648299105,3cyclictest1271756-21apt-key20:35:001
118648299104,4cyclictest1209078-21/usr/sbin/munin19:35:161
11864849997,1cyclictest0-21swapper/219:25:002
11864829991,7cyclictest1250670-21taskset20:15:121
11864789983,3cyclictest1186528-21latency_hist19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional