You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-10 - 07:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot3.osadl.org (updated Tue Oct 10, 2023 10:56:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
121398540,15rtkit-daemon400614-21irqstats07:25:291
4539852470,7chrt453988-21unixbench_singl09:45:432
121398470,21rtkit-daemon1-21systemd10:19:541
393979994435,3cyclictest0-21swapper/309:15:013
393979994334,3cyclictest0-21swapper/310:31:323
121398430,8rtkit-daemon1-21systemd09:49:311
121398430,4rtkit-daemon0-21swapper/107:20:301
121398430,16rtkit-daemon404640-21ps07:35:331
39397999423,34cyclictest452525-21turbostat09:49:563
39397999422,35cyclictest425858-21turbostat08:39:563
39397999422,34cyclictest414775-21turbostat08:09:553
39397999413,32cyclictest424013-21turbostat08:34:563
39397999409,24cyclictest420306-21turbostat08:24:563
39397999402,33cyclictest468096-21turbostat10:29:563
39397999402,31cyclictest460702-21turbostat10:09:563
393979993930,4cyclictest0-21swapper/309:35:253
39397999392,33cyclictest437215-21turbostat09:09:563
393979993916,16cyclictest4146002sleep308:03:083
39397999383,29cyclictest466230-21turbostat10:24:553
39397999383,29cyclictest395729-21turbostat07:19:553
393979993828,4cyclictest0-21swapper/309:26:383
39397999382,32cyclictest411070-21turbostat07:59:553
393979993812,22cyclictest393834-21turbostat07:14:553
121398380,13rtkit-daemon1-21systemd08:22:541
121398380,10rtkit-daemon1-21systemd09:10:191
393979993729,3cyclictest0-21swapper/307:26:313
393979993728,4cyclictest0-21swapper/310:49:413
393979993728,3cyclictest0-21swapper/307:30:313
121398370,15rtkit-daemon1-21systemd08:29:051
393979993627,3cyclictest0-21swapper/308:56:313
393979993627,3cyclictest0-21swapper/308:17:313
393979993626,4cyclictest0-21swapper/307:52:313
121398360,21rtkit-daemon1-21systemd07:08:221
121398360,15rtkit-daemon280-21plymouthd10:40:251
121398360,10rtkit-daemon1-21systemd08:48:411
393979993526,4cyclictest0-21swapper/308:26:313
393979993526,3cyclictest0-21swapper/310:14:363
393979993526,3cyclictest0-21swapper/309:31:323
393979993525,4cyclictest0-21swapper/308:14:563
393979993514,15cyclictest4739822sleep310:40:253
393979993511,16cyclictest4313852sleep308:46:473
121398350,7rtkit-daemon280-21plymouthd09:29:021
393979993426,4cyclictest0-21swapper/310:50:343
393979993426,3cyclictest0-21swapper/310:35:383
393979993425,3cyclictest0-21swapper/309:59:563
393979993425,3cyclictest0-21swapper/307:36:313
393979993424,4cyclictest0-21swapper/309:51:323
393979993424,3cyclictest0-21swapper/307:22:313
39397999342,26cyclictest431666-21turbostat08:54:553
393979993415,14cyclictest4417282sleep309:15:343
121398340,8rtkit-daemon1-21systemd09:09:171
4594672330,6sleep1301rcuc/110:00:251
393979993325,3cyclictest0-21swapper/310:15:253
393979993325,3cyclictest0-21swapper/309:40:403
393979993325,3cyclictest0-21swapper/309:22:333
393979993325,3cyclictest0-21swapper/309:22:333
121398330,9rtkit-daemon1-21systemd10:36:471
121398330,18rtkit-daemon1-21systemd08:16:150
393979993223,4cyclictest0-21swapper/307:45:173
393979993223,3cyclictest0-21swapper/310:04:313
393979993222,4cyclictest0-21swapper/308:43:153
3921972322,6sleep30-21swapper/307:05:033
121398320,9rtkit-daemon1-21systemd10:27:291
121398320,9rtkit-daemon1-21systemd07:34:361
121398320,9rtkit-daemon1-21systemd07:12:091
121398320,8rtkit-daemon1-21systemd10:12:522
121398320,8rtkit-daemon1-21systemd09:41:141
121398320,8rtkit-daemon1-21systemd07:50:461
121398320,17rtkit-daemon1-21systemd08:58:351
121398320,16rtkit-daemon280-21plymouthd10:07:092
121398310,8rtkit-daemon1-21systemd10:22:541
121398310,8rtkit-daemon1-21systemd09:59:462
121398310,8rtkit-daemon1-21systemd07:40:421
121398310,6rtkit-daemon280-21plymouthd10:50:391
121398310,17rtkit-daemon1-21systemd09:04:111
121398300,9rtkit-daemon1-21systemd09:53:381
121398300,9rtkit-daemon1-21systemd09:32:521
121398300,9rtkit-daemon1-21systemd08:54:121
121398300,8rtkit-daemon1-21systemd08:40:171
121398300,17rtkit-daemon411071-21perf07:59:501
121398300,15rtkit-daemon1-21systemd07:45:431
4227132296,6sleep2171rcu_preempt08:26:152
4118022294,9sleep2411870-21missed_timers07:55:332
393979992921,3cyclictest0-21swapper/307:40:313
121398290,9rtkit-daemon1-21systemd08:00:091
121398290,8rtkit-daemon1-21systemd09:35:441
121398290,8rtkit-daemon1-21systemd08:36:471
121398290,16rtkit-daemon1-21systemd07:15:551
121398290,14rtkit-daemon1-21systemd09:23:201
121398290,14rtkit-daemon1-21systemd09:23:201
393979992810,13cyclictest4369972sleep309:02:353
121398280,8rtkit-daemon1-21systemd10:16:572
121398280,8rtkit-daemon1-21systemd08:09:231
121398280,5rtkit-daemon0-21swapper/108:34:131
4413002273,7sleep0441355-21grep09:15:260
4342342273,16sleep0434241-21irqstats08:55:320
121398270,6rtkit-daemon280-21plymouthd09:19:251
121398270,4rtkit-daemon0-21swapper/108:11:541
121398270,13rtkit-daemon1217-21sendmail-mta10:33:321
4463522262,11sleep00-21swapper/009:29:090
4387092263,5sleep00-21swapper/009:05:540
121398260,4rtkit-daemon0-21swapper/210:00:212
4503632253,4sleep00-21swapper/009:39:550
4318972253,4sleep00-21swapper/008:50:100
4076862252,5sleep0407780-21telnet07:45:240
4058832242,4sleep00-21swapper/007:40:230
4033652242,4sleep00-21swapper/007:32:380
121398240,8rtkit-daemon280-21plymouthd10:49:271
4696712232,4sleep00-21swapper/010:26:590
4369222233,3sleep00-21swapper/009:01:350
4332412232,5sleep20-21swapper/208:51:562
4257012232,4sleep00-21swapper/008:33:250
4052422232,4sleep00-21swapper/007:36:480
3979832232,4sleep00-21swapper/007:19:470
3937572232,15sleep00-21swapper/007:09:350
4753142222,15sleep00-21swapper/010:43:110
4714942222,3sleep00-21swapper/010:31:410
4678742222,3sleep00-21swapper/010:22:330
4640802222,3sleep00-21swapper/010:11:270
4584522222,3sleep00-21swapper/009:58:490
4408062222,3sleep00-21swapper/009:14:140
4238072223,3sleep00-21swapper/008:27:460
4219252222,15sleep00-21swapper/008:22:150
4122172222,3sleep00-21swapper/007:55:380
4109562223,14sleep00-21swapper/007:53:570
3954172222,3sleep00-21swapper/007:11:230
4733352212,4sleep20-21swapper/210:36:422
4622852212,14sleep00-21swapper/010:07:060
4548572212,14sleep00-21swapper/009:50:080
4542202212,15sleep00-21swapper/009:48:310
4522102213,13sleep00-21swapper/009:41:200
4314322212,4sleep00-21swapper/008:47:240
4181752212,3sleep00-21swapper/008:11:420
4165472213,13sleep00-21swapper/008:09:290
4144952212,3sleep00-21swapper/008:01:450
4014302212,4sleep00-21swapper/007:26:310
3996912212,3sleep00-21swapper/007:22:490
4789942202,5sleep2280-21plymouthd10:52:502
4659612202,3sleep00-21swapper/010:16:580
4624722202,13sleep10-21swapper/110:09:311
4597452202,3sleep00-21swapper/010:00:340
4501492203,12sleep20-21swapper/209:37:122
4295162202,3sleep00-21swapper/008:41:250
4788772192,11sleep00-21swapper/010:51:240
4771742192,11sleep00-21swapper/010:48:270
4771642192,4sleep20-21swapper/210:48:202
4723972193,9sleep0472407-21diskstats10:35:270
4521972192,4sleep20-21swapper/209:41:112
4434032192,11sleep00-21swapper/009:20:330
4434032192,11sleep00-21swapper/009:20:330
4196042192,12sleep1419685-21cat08:15:371
3997132192,3sleep20-21swapper/207:23:082
4637022182,4sleep10-21swapper/110:10:481
4476672182,11sleep00-21swapper/009:30:340
4405872182,3sleep20-21swapper/209:11:222
4131702182,11sleep2689-21snmpd08:00:142
39517321810,4sleep20-21swapper/207:10:372
4584932172,3sleep10-21swapper/109:59:221
4260912172,11sleep00-21swapper/008:35:140
4184332172,10sleep2418630-21cut08:15:022
4677992162,4sleep20-21swapper/210:21:342
4433862161,10sleep2443398-21sshd09:20:312
4433862161,10sleep2443398-21sshd09:20:312
4427072162,3sleep20-21swapper/209:19:552
4387582162,9sleep20-21swapper/209:06:332
4371232162,3sleep20-21swapper/209:04:152
4295402162,3sleep20-21swapper/208:41:432
4350602152,9sleep20-21swapper/208:56:392
4275122152,8sleep20-21swapper/208:38:012
4109282152,3sleep20-21swapper/207:53:352
4091022152,7sleep20-21swapper/207:48:482
4069032152,6sleep2407007-21users07:40:362
3978212152,8sleep20-21swapper/207:17:442
4747912142,6sleep2474799-21python310:40:442
4693452143,5sleep2469406-21/usr/sbin/munin10:25:412
4562062145,5sleep2280-21plymouthd09:51:592
4462802143,5sleep2280-21plymouthd09:28:152
4315802142,8sleep20-21swapper/208:49:192
4257552142,7sleep20-21swapper/208:34:082
3919982142,5sleep20-21swapper/207:05:012
4701032132,5sleep20-21swapper/210:30:032
4484942132,5sleep20-21swapper/209:34:342
4218202133,5sleep2280-21plymouthd08:20:532
4149982132,5sleep20-21swapper/208:05:042
4032472132,5sleep20-21swapper/207:31:072
4007932132,5sleep2400829-21ntp_states07:25:322
4201852122,4sleep20-21swapper/208:18:512
4038502112,4sleep20-21swapper/207:35:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional