You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 21:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 23, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15274992827,1cyclictest13-21ksoftirqd/009:35:120
1527499260,24cyclictest0-21swapper/010:40:150
15279992523,1cyclictest0-21swapper/109:22:541
1527999250,24cyclictest9015-21diskmemload11:52:521
1527999250,24cyclictest0-21swapper/110:00:081
1527999250,24cyclictest0-21swapper/109:54:421
1527999250,1cyclictest0-21swapper/110:54:011
1527499250,24cyclictest0-21swapper/011:57:180
1527499250,24cyclictest0-21swapper/010:37:590
1527499250,23cyclictest0-21swapper/009:46:270
15289992423,1cyclictest25855-21ssh09:53:183
1528999240,2cyclictest25594-21chrt11:48:123
1528999240,1cyclictest6809-21ssh11:32:453
15284992422,1cyclictest6194-21sh09:36:432
15284992419,0cyclictest0-21swapper/212:13:202
1528499240,24cyclictest30789-21ssh10:27:422
1528499240,23cyclictest9518-21sh11:05:332
1528499240,23cyclictest9015-21diskmemload10:12:472
1528499240,23cyclictest0-21swapper/211:37:142
15279992423,1cyclictest9015-21diskmemload09:10:411
15279992423,1cyclictest0-21swapper/112:25:291
1527999240,24cyclictest0-21swapper/111:05:481
1527999240,23cyclictest9052-21rm09:38:161
1527999240,23cyclictest25399-21/usr/sbin/munin10:23:011
1527999240,23cyclictest16101-21sh10:13:371
1527999240,23cyclictest0-21swapper/112:30:271
1527999240,0cyclictest0-21swapper/111:31:241
15274992423,0cyclictest9015-21diskmemload10:31:190
15274992420,3cyclictest23285-21rm10:20:000
1527499240,24cyclictest0-21swapper/011:07:570
1527499240,23cyclictest9015-21diskmemload11:30:250
1527499240,23cyclictest0-21swapper/012:32:550
1527499240,23cyclictest0-21swapper/012:25:380
1527499240,23cyclictest0-21swapper/012:15:470
15289992323,0cyclictest0-21swapper/310:58:243
15289992323,0cyclictest0-21swapper/310:27:423
15289992323,0cyclictest0-21swapper/309:48:323
15289992322,1cyclictest9015-21diskmemload10:47:003
15289992322,1cyclictest7563-21ssh12:01:373
15289992322,1cyclictest19726-21sh10:17:503
15289992322,1cyclictest11976-21ssh11:37:143
15289992321,2cyclictest3711-21ssh11:28:433
15289992321,2cyclictest3711-21ssh11:28:433
15289992320,2cyclictest12927-21ssh10:11:413
1528999230,2cyclictest611-21irqbalance11:20:033
1528999230,2cyclictest14840-21sh11:10:023
1528999230,23cyclictest9015-21diskmemload09:16:123
1528999230,23cyclictest0-21swapper/311:58:183
1528999230,23cyclictest0-21swapper/310:32:543
1528999230,23cyclictest0-21swapper/309:23:293
1528999230,23cyclictest0-21swapper/309:10:313
1528999230,22cyclictest9161-21ssh09:38:243
1528999230,22cyclictest27872-21ssh11:50:543
1528999230,0cyclictest0-21swapper/312:27:453
15284992323,0cyclictest9015-21diskmemload11:50:442
15284992323,0cyclictest0-21swapper/211:55:042
15284992323,0cyclictest0-21swapper/211:42:102
15284992323,0cyclictest0-21swapper/211:31:242
15284992323,0cyclictest0-21swapper/210:00:082
15284992323,0cyclictest0-21swapper/209:53:182
15284992323,0cyclictest0-21swapper/209:44:262
15284992322,1cyclictest9015-21diskmemload10:36:132
15284992322,0cyclictest0-21swapper/209:23:522
15284992321,1cyclictest26927-21awk11:50:002
1528499230,23cyclictest9015-21diskmemload09:29:402
1528499230,23cyclictest0-21swapper/212:29:142
1528499230,23cyclictest0-21swapper/211:17:022
1528499230,23cyclictest0-21swapper/209:10:342
1528499230,22cyclictest28515-21ssh09:56:452
1528499230,22cyclictest14023-21sh12:36:092
1528499230,0cyclictest0-21swapper/212:18:282
15279992323,0cyclictest0-21swapper/111:59:391
15279992323,0cyclictest0-21swapper/111:16:521
15279992321,2cyclictest29319-21sh12:20:431
15279992321,1cyclictest7238-21sh10:06:301
15279992321,1cyclictest0-21swapper/109:15:341
1527999230,23cyclictest0-21swapper/112:35:471
1527999230,23cyclictest0-21swapper/112:00:211
1527999230,23cyclictest0-21swapper/110:42:391
1527999230,23cyclictest0-21swapper/110:15:571
1527999230,22cyclictest9015-21diskmemload10:28:111
1527999230,22cyclictest26585-21cron08:40:001
1527999230,22cyclictest15706-21sh11:11:161
1527999230,22cyclictest0-21swapper/111:50:001
1527999230,22cyclictest0-21swapper/111:23:191
1527999230,22cyclictest0-21swapper/109:48:121
1527999230,22cyclictest0-21swapper/109:42:551
1527999230,22cyclictest0-21swapper/109:07:531
1527999230,1cyclictest0-21swapper/111:28:431
1527999230,1cyclictest0-21swapper/111:28:431
1527999230,0cyclictest0-21swapper/109:59:391
15274992323,0cyclictest9015-21diskmemload09:20:590
15274992323,0cyclictest0-21swapper/012:24:360
15274992323,0cyclictest0-21swapper/010:00:510
15274992323,0cyclictest0-21swapper/009:42:340
15274992323,0cyclictest0-21swapper/009:25:170
15274992323,0cyclictest0-21swapper/009:19:220
15274992321,1cyclictest0-21swapper/012:36:030
15274992321,1cyclictest0-21swapper/011:51:120
15274992320,2cyclictest12793-21sh12:06:130
1527499230,23cyclictest0-21swapper/011:10:330
1527499230,22cyclictest9287-21sh09:10:030
1527499230,22cyclictest9015-21diskmemload11:35:150
1527499230,22cyclictest22493-21sh09:50:590
1527499230,22cyclictest22440-21latency_hist07:25:000
1527499230,22cyclictest16750-21sh11:41:000
1527499230,1cyclictest0-21swapper/011:45:030
1527499230,0cyclictest9015-21diskmemload10:50:090
1527499230,0cyclictest0-21swapper/010:56:520
1527499230,0cyclictest0-21swapper/009:56:220
15289992222,0cyclictest9015-21diskmemload12:18:283
15289992222,0cyclictest9015-21diskmemload09:57:063
15289992222,0cyclictest0-21swapper/312:30:023
15289992222,0cyclictest0-21swapper/311:17:023
15289992222,0cyclictest0-21swapper/310:04:163
15289992222,0cyclictest0-21swapper/309:29:403
15289992220,1cyclictest31397-21turbostat08:50:003
1528999220,22cyclictest12292-21ssh10:38:553
1528999220,22cyclictest0-21swapper/312:10:143
1528999220,22cyclictest0-21swapper/309:31:553
1528999220,21cyclictest9215-21ssh11:05:013
1528999220,21cyclictest6593-21ssh10:05:403
1528999220,21cyclictest4331-21apt-get09:02:543
1528999220,21cyclictest21398-21apt-get08:30:023
1528999220,21cyclictest0-21swapper/310:42:553
1528999220,21cyclictest0-21swapper/307:25:003
1528999220,1cyclictest26302-21sh10:51:573
15284992222,0cyclictest0-21swapper/212:30:062
15284992222,0cyclictest0-21swapper/212:21:422
15284992222,0cyclictest0-21swapper/212:07:562
15284992220,1cyclictest17099-21apt-get09:17:552
15284992220,1cyclictest15647-21ssh11:11:122
1528499220,22cyclictest0-21swapper/211:26:502
1528499220,22cyclictest0-21swapper/211:26:502
1528499220,22cyclictest0-21swapper/210:47:002
1528499220,21cyclictest9015-21diskmemload10:05:432
1528499220,21cyclictest9015-21diskmemload09:30:192
1528499220,21cyclictest24588-21rm10:21:432
1528499220,21cyclictest17102-21apt-get10:15:012
1528499220,1cyclictest4571-21ssh11:01:192
1528499220,1cyclictest27016-21ssh11:21:312
1528499220,1cyclictest14754-21taskset10:41:582
1528499220,0cyclictest0-21swapper/212:01:372
15279992222,0cyclictest0-21swapper/112:05:301
15279992222,0cyclictest0-21swapper/111:43:571
15279992222,0cyclictest0-21swapper/111:35:191
15279992222,0cyclictest0-21swapper/110:35:241
15279992220,2cyclictest20028-21awk07:20:001
15279992220,1cyclictest0-21swapper/109:30:441
1527999220,21cyclictest9015-21diskmemload10:30:541
1527999220,21cyclictest14660-21apt-get08:17:531
1527999220,21cyclictest0-21swapper/112:10:191
1527999220,21cyclictest0-21swapper/110:55:521
1527999220,21cyclictest0-21swapper/110:45:101
1527999220,1cyclictest0-21swapper/109:25:501
1527999220,0cyclictest0-21swapper/112:15:411
1527999220,0cyclictest0-21swapper/111:00:211
15274992222,0cyclictest9015-21diskmemload12:12:070
15274992222,0cyclictest0-21swapper/009:31:560
15274992222,0cyclictest0-21swapper/008:47:530
15274992221,1cyclictest62550irq/127-eno110:10:070
15274992221,1cyclictest13-21ksoftirqd/011:23:150
15274992221,0cyclictest27086-21apt-get08:42:530
15274992220,1cyclictest30983-21apt-get10:27:550
15274992220,1cyclictest0-21swapper/011:25:030
15274992220,1cyclictest0-21swapper/011:25:030
15274992220,1cyclictest0-21swapper/010:05:480
15274992220,1cyclictest0-21swapper/008:40:000
1527499221,21cyclictest15891-21kworker/0:008:12:140
1527499220,22cyclictest0-21swapper/010:45:580
1527499220,21cyclictest11786-21turbostat08:10:000
1527499220,21cyclictest0-21swapper/011:15:340
1527499220,21cyclictest0-21swapper/011:00:460
15289992120,1cyclictest25316-21apt-get07:32:533
15289992120,1cyclictest13539-21ssh12:07:103
1528999210,21cyclictest9015-21diskmemload09:41:033
1528999210,21cyclictest0-21swapper/311:40:103
1528999210,20cyclictest18109-21apt-get07:17:553
1528999210,0cyclictest0-21swapper/312:21:333
1528999210,0cyclictest0-21swapper/311:00:513
1528999210,0cyclictest0-21swapper/310:20:083
1528999210,0cyclictest0-21swapper/308:37:553
15284992120,1cyclictest0-21swapper/210:55:522
1528499210,0cyclictest0-21swapper/209:05:022
15279992121,0cyclictest0-21swapper/108:27:541
15279992120,1cyclictest32681-21apt-get07:47:551
15279992120,0cyclictest0-21swapper/107:52:541
1527999210,21cyclictest0-21swapper/108:47:521
1527999210,0cyclictest0-21swapper/107:22:541
15274992119,1cyclictest6244-21apt-get09:05:020
1527499210,1cyclictest0-21swapper/008:00:000
1527499210,1cyclictest0-21swapper/007:47:550
15289992020,0cyclictest0-21swapper/309:05:023
15289992019,1cyclictest15708-21apt-get07:12:533
15284992020,0cyclictest29479-21apt-get08:47:522
15284992020,0cyclictest0-21swapper/208:17:532
15284992020,0cyclictest0-21swapper/208:12:542
15284992020,0cyclictest0-21swapper/207:17:552
15284992019,0cyclictest20528-21apt-get07:22:542
1528499200,0cyclictest22275-21apt-get08:32:542
1528499200,0cyclictest0-21swapper/208:22:532
1528499200,0cyclictest0-21swapper/207:32:532
15279992019,1cyclictest0-21swapper/107:42:531
15279992019,0cyclictest0-21swapper/107:12:531
15274992020,0cyclictest0-21swapper/008:22:520
15274992019,0cyclictest0-21swapper/007:12:530
1527499201,19cyclictest15891-21kworker/0:008:53:060
1527499200,0cyclictest0-21swapper/007:42:530
1527499200,0cyclictest0-21swapper/007:32:530
15289991919,0cyclictest0-21swapper/307:27:533
15289991918,1cyclictest0-21swapper/308:17:533
15284991919,0cyclictest0-21swapper/207:47:552
1528499190,1cyclictest6224-21cron09:05:002
15279991919,0cyclictest0-21swapper/108:57:561
15279991919,0cyclictest0-21swapper/108:30:021
15279991919,0cyclictest0-21swapper/107:57:551
15279991918,1cyclictest9868-21apt-get08:07:541
15279991918,1cyclictest25316-21apt-get07:32:531
15274991917,1cyclictest9375-21cron08:05:000
15289991818,0cyclictest0-21swapper/308:57:563
15289991818,0cyclictest0-21swapper/308:12:543
15284991817,1cyclictest0-21swapper/207:42:532
15279991818,0cyclictest0-21swapper/108:52:531
1527999180,0cyclictest0-21swapper/109:02:541
15274991818,0cyclictest0-21swapper/008:30:020
15274991818,0cyclictest0-21swapper/008:27:540
15289991717,0cyclictest0-21swapper/308:27:543
1528999170,0cyclictest31928-21apt-get08:52:543
1528999170,0cyclictest0-21swapper/308:07:543
15284991717,0cyclictest0-21swapper/207:52:542
15284991717,0cyclictest0-21swapper/207:12:532
1528499170,1cyclictest3849-21perf09:00:002
1528499170,0cyclictest24671-21apt-get08:37:552
15279991717,0cyclictest0-21swapper/108:12:531
15274991715,1cyclictest16622-21idleruntime-cro08:20:010
15289991616,0cyclictest0-21swapper/308:22:533
15289991616,0cyclictest0-21swapper/307:47:553
15289991615,1cyclictest32172-21cron07:45:013
15289991615,1cyclictest2663-21apt-get07:52:543
15284991616,0cyclictest5061-21apt-get07:57:552
15284991614,0cyclictest151rcu_preempt08:07:542
1528499160,0cyclictest0-21swapper/207:26:362
15274991616,0cyclictest22920-21apt-get07:27:530
15274991616,0cyclictest0-21swapper/007:52:540
15274991615,0cyclictest0-21swapper/008:57:560
1527499160,15cyclictest19157-21ntp_states07:18:000
15284991515,0cyclictest0-21swapper/208:52:532
15289991414,0cyclictest0-21swapper/308:00:163
1528999140,14cyclictest0-21swapper/308:41:283
1528999140,0cyclictest0-21swapper/307:55:033
1528999140,0cyclictest0-21swapper/307:35:123
15284991414,0cyclictest0-21swapper/208:00:322
1528499140,14cyclictest0-21swapper/208:40:342
1528499140,14cyclictest0-21swapper/207:35:512
15279991414,0cyclictest0-21swapper/108:20:121
15279991414,0cyclictest0-21swapper/107:35:031
1527999140,14cyclictest0-21swapper/108:40:381
1527999140,14cyclictest0-21swapper/107:25:061
1527999140,0cyclictest0-21swapper/108:00:091
1527499140,0cyclictest0-21swapper/009:01:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional