You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 22:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Sep 23, 2026 12:42:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3550447998171,6cyclictest3589358-40mandb09:14:581
3550446997159,9cyclictest3589358-40mandb09:15:200
3550448997054,10cyclictest3589358-40mandb09:14:552
3550446996552,9cyclictest3589358-40mandb09:14:470
3550447996351,7cyclictest3636529-21ssh10:31:251
3550448995742,6cyclictest0-21swapper/209:15:462
3550447995431,19cyclictest3672333-21aten_r2power_vo11:31:091
3550446995417,32cyclictest3691053-21diskmemload12:01:170
3550448995236,6cyclictest0-21swapper/210:06:222
3550447995241,7cyclictest3665706-21cat11:20:021
355044799522,43cyclictest3669770-21ssh11:26:141
3550448995112,5cyclictest171rcu_preempt07:46:182
3550447995142,6cyclictest3589837-21ssh09:15:321
3550447995140,6cyclictest439-21systemd-logind09:00:021
3550447995136,8cyclictest0-21swapper/108:46:211
3550447995128,20cyclictest1002-21mta-sts-daemon12:34:311
3550447994926,20cyclictest3620078-21latency_hist10:05:021
3550447994619,23cyclictest3561459-21fschecks_count07:46:091
355044799461,22cyclictest3678955-21ssh11:41:171
355044699462,35cyclictest3684962-21ssh11:51:200
3550447994521,21cyclictest0-21swapper/110:21:301
3550447994520,21cyclictest3614613-21aten_r2power_vo09:56:071
3550446994528,14cyclictest3609637-21unixbench_multi09:46:270
3550447994435,5cyclictest3706219-21needreboot12:26:141
3550447994435,5cyclictest3677653-21latency_hist11:40:011
3550447994419,3cyclictest3641240-21kworker/u8:5-xprtiod10:57:361
3550447994416,7cyclictest3627844-21sed10:16:281
3550448994334,5cyclictest0-21swapper/209:41:172
3550448994334,5cyclictest0-21swapper/209:41:172
3550447994321,20cyclictest0-21swapper/111:56:061
355044699430,36cyclictest3703288-21ssh12:21:170
3550448994234,5cyclictest3699896-21if_wlan012:16:112
3550448994232,6cyclictest3595700-21latency_hist09:25:022
3550448994225,10cyclictest351ktimers/208:41:182
3550448994218,4cyclictest171rcu_preempt11:36:182
3550447994236,3cyclictest0-21swapper/112:20:001
3550447994235,4cyclictest3554841-21unixbench_multi07:21:271
355044799422,16cyclictest3609053-21ntpq09:46:141
3550447994221,18cyclictest0-21swapper/111:55:011
3550447994219,19cyclictest0-21swapper/110:41:161
3550447994217,5cyclictest2763350-21cups-browsed08:14:021
3550446994234,5cyclictest420-21avahi-daemon11:39:510
355044699422,33cyclictest3612263-21ssh09:51:250
3550446994223,11cyclictest151ktimers/011:16:170
355044899413,32cyclictest3691128-21ssh12:01:202
3550447994121,17cyclictest1001-21mta-sts-daemon10:35:361
3550446994132,6cyclictest151ktimers/009:56:190
3550448994029,8cyclictest1003-21mta-sts-daemon11:05:462
355044799404,9cyclictest428-21mta-sts-daemon11:23:361
355044799403,10cyclictest3712456-21ssh12:36:201
3550447994020,17cyclictest0-21swapper/111:50:011
3550447994014,7cyclictest3565416-21idleruntime-cro08:00:011
3550447994012,9cyclictest1000-21mta-sts-daemon08:16:181
3550446994035,3cyclictest0-21swapper/009:26:230
355044699403,31cyclictest151ktimers/008:26:170
3550446994029,8cyclictest3649060-21timerwakeupswit10:51:310
3550446994019,13cyclictest151ktimers/008:11:210
3550448993930,6cyclictest3711124-21unin-run12:35:012
3550448993928,7cyclictest3599225-21acpi09:31:072
3550447993933,3cyclictest1000-21mta-sts-daemon09:31:081
355044799393,10cyclictest3657635-21ssh11:06:181
3550447993929,6cyclictest3559478-21latency_hist07:40:021
3550447993917,3cyclictest0-21swapper/109:51:091
3550446993931,5cyclictest2763350-21cups-browsed11:34:540
3550448993831,4cyclictest3653347-21cron11:00:012
3550448993830,5cyclictest3571424-21ls08:20:002
3550448993826,9cyclictest420-21avahi-daemon07:31:162
355044899381,6cyclictest3697316-21ssh12:11:262
355044899381,6cyclictest3697316-21ssh12:11:262
355044799382,12cyclictest3648628-21ssh10:51:221
3550447993819,11cyclictest3645553-21ssh10:46:181
3550447993810,8cyclictest3660433-21/usr/sbin/munin11:11:161
3550446993831,4cyclictest3619976-21kworker/u8:10-xprtiod10:05:010
3550446993829,6cyclictest421-21dbus-daemon10:22:490
3550446993823,13cyclictest3582445-21fschecks_time08:56:120
3550448993730,4cyclictest0-21swapper/211:11:322
355044899372,26cyclictest3612227-21ssh09:51:232
355044799375,13cyclictest3606052-21ntpq09:41:161
355044799375,13cyclictest3606052-21ntpq09:41:161
355044799375,11cyclictest3654754-21ssh11:01:231
3550447993729,5cyclictest3553423-21cron07:20:001
3550447993729,5cyclictest3553423-21cron07:20:001
3550447993728,4cyclictest0-21swapper/112:21:221
3550447993727,6cyclictest2763350-21cups-browsed10:12:081
3550447993714,2cyclictest0-21swapper/109:30:031
355044699373,8cyclictest3633459-21ssh10:26:200
3550446993728,5cyclictest1003-21mta-sts-daemon07:51:150
3550446993723,7cyclictest12750irq/33-eth011:01:180
3550448993627,4cyclictest0-21swapper/210:31:262
355044899362,23cyclictest3630314-21ssh10:21:182
355044799368,20cyclictest1000-21mta-sts-daemon09:06:191
3550447993627,6cyclictest3602344-21aten_r2power_en09:36:061
3550447993627,6cyclictest3602344-21aten_r2power_en09:36:061
3550446993626,7cyclictest3603346-21switchtime09:36:270
3550446993626,7cyclictest3603346-21switchtime09:36:270
355044699362,28cyclictest3709457-21ssh12:31:260
3550448993527,5cyclictest3586446-21idleruntime-cro09:10:012
3550448993526,6cyclictest428-21mta-sts-daemon08:31:172
3550448993526,6cyclictest3556451-21systemd-run07:30:022
3550448993525,6cyclictest420-21avahi-daemon12:27:262
3550448993525,4cyclictest1001-21mta-sts-daemon08:01:132
355044799356,18cyclictest428-21mta-sts-daemon08:06:181
355044799356,18cyclictest428-21mta-sts-daemon08:06:181
3550447993527,5cyclictest1003-21mta-sts-daemon08:26:191
355044799352,10cyclictest3694194-21ssh12:06:181
3550447993513,3cyclictest0-21swapper/110:26:241
3550446993527,5cyclictest3697522-21switchtime12:11:310
3550446993527,5cyclictest3697522-21switchtime12:11:310
3550446993527,4cyclictest0-21swapper/012:20:000
3550446993526,7cyclictest3639983-21unixbench_multi10:36:290
3550448993427,4cyclictest3569480-21processes08:11:232
3550448993427,4cyclictest3564738-21needreboot07:56:172
3550448993427,3cyclictest0-21swapper/210:01:322
3550448993427,3cyclictest0-21swapper/210:00:012
3550448993426,5cyclictest3665715-21latency_hist11:20:022
355044899342,27cyclictest3685058-21ssh11:51:232
355044899342,25cyclictest3654790-21ssh11:01:252
3550447993426,5cyclictest3593201-21aten_r2power_fr09:21:071
3550447993425,6cyclictest439-21systemd-logind07:15:011
3550447993425,6cyclictest439-21systemd-logind07:15:011
3550446993427,5cyclictest485-21in:imuxsock08:00:010
3550446993426,4cyclictest0-21swapper/011:26:160
3550446993413,17cyclictest3636426-21ssh10:31:210
3550448993327,3cyclictest0-21swapper/207:20:002
3550448993327,3cyclictest0-21swapper/207:20:002
3550448993326,3cyclictest0-21swapper/211:41:222
3550448993325,5cyclictest265-21systemd-journal09:50:012
3550448993323,5cyclictest0-21swapper/208:46:152
3550448993321,8cyclictest420-21avahi-daemon10:20:012
355044899331,25cyclictest3666952-21ssh11:21:232
3550447993329,3cyclictest3550445-21cyclictest07:51:161
3550447993325,5cyclictest3575813-21grep08:31:281
3550447993324,5cyclictest3572960-21systemd-run08:25:011
3550446993324,5cyclictest1003-21mta-sts-daemon07:21:150
3550446993322,8cyclictest3577413-21cron08:40:010
3550446993321,5cyclictest1002-21mta-sts-daemon11:11:340
3550446993320,3cyclictest0-21swapper/009:41:070
3550446993320,3cyclictest0-21swapper/009:41:070
3550448993226,4cyclictest57-21kcompactd010:26:202
3550448993224,5cyclictest3603302-21switchtime09:36:272
3550448993224,5cyclictest3603302-21switchtime09:36:272
3550448993224,5cyclictest3581918-21sh08:55:012
3550448993224,4cyclictest171rcu_preempt08:06:232
3550448993224,4cyclictest171rcu_preempt08:06:232
3550448993223,5cyclictest3689822-21sendmail12:00:012
3550448993221,4cyclictest3595918-21ssh09:25:232
3550448993216,3cyclictest171rcu_preempt08:21:172
3550447993225,4cyclictest1000-21mta-sts-daemon10:05:421
3550447993223,5cyclictest3581942-21systemd-run08:55:011
3550446993227,3cyclictest3623865-21fschecks_count10:11:120
3550446993226,3cyclictest0-21swapper/011:40:250
3550446993226,3cyclictest0-21swapper/009:34:310
355044699321,27cyclictest3657261-21diskstats11:06:110
3550448993123,5cyclictest3714123-21munin-run12:40:012
355044899312,21cyclictest2763347-21cupsd08:26:182
3550448993121,6cyclictest3563946-21systemd-run07:55:012
3550448993118,9cyclictest0-21swapper/207:40:012
355044799313,10cyclictest428-21mta-sts-daemon12:03:131
3550446993127,2cyclictest0-21swapper/012:40:010
3550446993124,4cyclictest3644900-21aten_r2power_en10:46:060
3550446993124,4cyclictest3557863-21head07:31:290
355044699312,22cyclictest2763350-21cups-browsed07:46:190
3550446993121,7cyclictest3586430-21diskmemload09:10:010
3550446993119,9cyclictest0-21swapper/008:35:020
3550448993022,5cyclictest3674613-21cron11:35:012
3550448993021,6cyclictest1001-21mta-sts-daemon11:29:102
3550448993021,3cyclictest0-21swapper/212:21:222
3550448993014,11cyclictest421-21dbus-daemon10:38:562
355044899301,22cyclictest3624310-21sh10:11:212
3550446993024,4cyclictest3695910-21systemd-run12:10:020
3550446993024,3cyclictest0-21swapper/012:26:220
3550446993024,3cyclictest0-21swapper/009:21:090
3550446993023,4cyclictest3688248-21sendmail_mailst11:56:230
3550446993023,4cyclictest3579086-21latency_hist08:45:020
3550446993023,3cyclictest0-21swapper/010:41:220
3550446993018,9cyclictest3682254-21mailstats11:46:290
3550446993016,11cyclictest12650irq/32-eth010:06:200
355044899295,3cyclictest171rcu_preempt10:51:232
3550448992923,3cyclictest0-21swapper/209:00:012
3550448992922,4cyclictest0-21swapper/212:09:362
3550448992921,5cyclictest3551846-21unixbench_singl07:11:282
3550448992921,5cyclictest3551846-21unixbench_singl07:11:282
3550448992920,5cyclictest1003-21mta-sts-daemon10:44:492
3550448992911,3cyclictest171rcu_preempt10:46:192
3550447992923,3cyclictest0-21swapper/108:45:011
3550447992923,3cyclictest0-21swapper/107:30:021
3550446992921,6cyclictest3560458-21proc_pri07:41:220
3550446992921,5cyclictest2763350-21cups-browsed08:20:000
355044699291,14cyclictest3551399-21ntpq07:11:200
355044699291,14cyclictest3551399-21ntpq07:11:200
3550448992819,6cyclictest3560933-21cron07:45:012
3550447992821,4cyclictest1002-21mta-sts-daemon12:11:531
3550447992821,4cyclictest1002-21mta-sts-daemon12:11:531
355044699282,20cyclictest3627461-21ssh10:16:220
355044899273,14cyclictest3682083-21ssh11:46:252
3550448992719,5cyclictest1-21systemd07:10:012
3550448992719,5cyclictest1-21systemd07:10:012
3550448992717,7cyclictest3584824-21grep09:01:282
355044799275,17cyclictest3577421-21munin-run08:40:011
3550447992719,5cyclictest3556968-21fschecks_time07:31:101
3550447992719,5cyclictest2763350-21cups-browsed09:01:551
3550446992721,4cyclictest3572934-21cron08:25:000
3550446992718,6cyclictest3558449-21fschecks_count07:36:090
3550446992717,7cyclictest0-21swapper/011:21:090
355044899266,3cyclictest171rcu_preempt08:36:172
3550448992612,3cyclictest171rcu_preempt07:21:142
3550447992621,2cyclictest0-21swapper/108:05:011
3550447992517,4cyclictest0-21swapper/107:41:161
3550446992512,10cyclictest0-21swapper/008:01:160
355044699242,14cyclictest151ktimers/008:46:200
3550446992416,6cyclictest3653360-21munin-run11:00:010
355044699241,18cyclictest3555863-21awk07:26:170
3550446992315,5cyclictest567-21gdbus08:10:010
3550446992315,5cyclictest567-21gdbus08:10:010
3550446992312,7cyclictest2763347-21cupsd08:51:180
3550446992217,3cyclictest151ktimers/007:16:150
3550446992217,3cyclictest151ktimers/007:16:150
3550446992115,3cyclictest151ktimers/009:01:170
355044999203,9cyclictest0-21swapper/309:15:033
355044999195,8cyclictest0-21swapper/309:21:203
355044999194,9cyclictest0-21swapper/309:15:103
355044999192,9cyclictest0-21swapper/310:51:263
3550447991913,3cyclictest2-21kthreadd07:10:011
3550447991913,3cyclictest2-21kthreadd07:10:011
355044699196,11cyclictest3550445-21cyclictest07:10:010
355044699196,11cyclictest3550445-21cyclictest07:10:010
355044999183,7cyclictest0-21swapper/310:01:253
355044999183,7cyclictest0-21swapper/307:21:223
355044999182,9cyclictest0-21swapper/311:31:203
355044999182,9cyclictest0-21swapper/311:16:253
355044999182,9cyclictest0-21swapper/309:31:213
355044999173,7cyclictest0-21swapper/312:01:173
355044999173,6cyclictest0-21swapper/309:51:183
355044999172,8cyclictest0-21swapper/310:21:203
355044999165,7cyclictest0-21swapper/310:41:073
355044999163,6cyclictest0-21swapper/310:16:143
355044999162,9cyclictest0-21swapper/311:51:213
355044999161,9cyclictest0-21swapper/311:01:173
355044999152,8cyclictest0-21swapper/312:16:113
355044999152,8cyclictest0-21swapper/310:00:023
355044999152,7cyclictest0-21swapper/312:36:173
355044999152,7cyclictest0-21swapper/311:36:183
355044999151,8cyclictest0-21swapper/309:36:173
355044999151,8cyclictest0-21swapper/309:36:173
355044999144,7cyclictest0-21swapper/310:31:203
355044999144,7cyclictest0-21swapper/309:26:163
355044999143,8cyclictest0-21swapper/308:31:183
355044999143,6cyclictest0-21swapper/312:31:223
355044999142,8cyclictest0-21swapper/312:21:143
355044999142,8cyclictest0-21swapper/311:11:153
355044999142,7cyclictest0-21swapper/309:46:153
355044999142,7cyclictest0-21swapper/309:05:013
355044999141,9cyclictest0-21swapper/312:06:083
355044999141,8cyclictest0-21swapper/312:11:163
355044999141,8cyclictest0-21swapper/312:11:163
355044999141,6cyclictest0-21swapper/308:51:163
355044999141,6cyclictest0-21swapper/308:16:163
355044999141,6cyclictest0-21swapper/308:01:153
355044999134,6cyclictest0-21swapper/310:06:153
355044999133,7cyclictest0-21swapper/308:26:153
355044999133,6cyclictest0-21swapper/311:56:053
355044999133,5cyclictest0-21swapper/308:36:143
355044999132,8cyclictest0-21swapper/310:55:363
355044999132,7cyclictest0-21swapper/311:41:173
355044999132,6cyclictest0-21swapper/310:26:153
355044999132,6cyclictest0-21swapper/307:46:183
355044999131,8cyclictest0-21swapper/311:46:083
355044999131,8cyclictest0-21swapper/311:26:163
355044999131,8cyclictest0-21swapper/311:21:143
355044999131,8cyclictest0-21swapper/307:40:013
355044999131,7cyclictest0-21swapper/312:26:063
355044999131,7cyclictest0-21swapper/311:06:063
355044999131,7cyclictest0-21swapper/310:11:133
355044999131,6cyclictest0-21swapper/309:41:053
355044999131,6cyclictest0-21swapper/309:41:053
355044999131,6cyclictest0-21swapper/309:06:183
355044999131,6cyclictest0-21swapper/308:21:193
355044999131,6cyclictest0-21swapper/307:26:173
355044999122,7cyclictest0-21swapper/310:50:013
355044999122,6cyclictest0-21swapper/308:56:253
355044999122,4cyclictest0-21swapper/307:41:143
355044999121,7cyclictest0-21swapper/308:41:233
355044999121,7cyclictest0-21swapper/307:35:023
355044999121,6cyclictest0-21swapper/310:35:383
355044999121,6cyclictest0-21swapper/308:06:213
355044999121,6cyclictest0-21swapper/308:06:213
355044999121,6cyclictest0-21swapper/307:11:153
355044999121,6cyclictest0-21swapper/307:11:153
355044999120,6cyclictest0-21swapper/308:11:223
355044999112,5cyclictest0-21swapper/307:51:153
355044999101,6cyclictest0-21swapper/307:16:183
355044999101,6cyclictest0-21swapper/307:16:183
355044999101,6cyclictest0-21swapper/307:10:013
355044999101,6cyclictest0-21swapper/307:10:013
355044999101,5cyclictest0-21swapper/308:46:083
355044999101,5cyclictest0-21swapper/307:56:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional