You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 19:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Mon Apr 13, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
303038996955,9cyclictest483963-21ssh12:10:391
30303799615,40cyclictest495221-21ssh12:25:420
30303799615,40cyclictest495221-21ssh12:25:420
30303799613,49cyclictest498770-21ssh12:30:370
30303799613,49cyclictest498770-21ssh12:30:370
303039995938,16cyclictest420-21avahi-daemon11:25:402
303038995934,21cyclictest416740-21fschecks_count10:45:301
303038995934,21cyclictest416740-21fschecks_count10:45:301
303039995844,5cyclictest0-21swapper/209:20:372
303039995844,5cyclictest0-21swapper/209:20:372
303039995340,8cyclictest420-21avahi-daemon11:04:112
303038995345,3cyclictest0-21swapper/111:20:011
303038995338,11cyclictest373125-21switchtime09:50:471
303038995327,4cyclictest411944-21awk10:40:011
30303999523,41cyclictest380855-21ssh10:00:402
303037995236,10cyclictest0-21swapper/009:45:460
303037995236,10cyclictest0-21swapper/009:45:460
303038995040,6cyclictest331551-21systemd-run08:45:011
303038994924,5cyclictest397287-21mailstats10:20:441
303037994941,5cyclictest421-21dbus-daemon08:35:010
303039994839,6cyclictest421-21dbus-daemon12:15:012
303038994838,6cyclictest0-21swapper/112:35:411
30303999474,32cyclictest372705-21ssh09:50:362
303039994738,6cyclictest482761-21ssh12:10:012
303039994736,6cyclictest0-21swapper/209:35:012
303038994723,4cyclictest0-21swapper/112:30:411
303038994723,4cyclictest0-21swapper/112:30:411
30303799471,40cyclictest441444-21ssh11:15:420
303038994638,5cyclictest383696-21idleruntime-cro10:05:011
303037994639,4cyclictest306064-21systemd-run07:20:010
303037994636,6cyclictest421-21dbus-daemon10:35:340
303039994538,3cyclictest0-21swapper/210:20:342
303039994533,4cyclictest0-21swapper/210:40:412
30303899452,36cyclictest468735-21ssh11:50:411
303039994434,6cyclictest421-21dbus-daemon10:00:012
303037994435,6cyclictest395828-21awk10:20:010
303037994435,6cyclictest395828-21awk10:20:010
303037994419,17cyclictest151ktimers/007:30:390
303039994334,6cyclictest475143-21true12:00:012
303038994335,5cyclictest424010-21systemd-run10:55:011
303038994335,5cyclictest424010-21systemd-run10:55:011
303037994337,4cyclictest0-21swapper/011:00:000
303039994235,3cyclictest0-21swapper/212:15:362
303039994233,6cyclictest459886-21true11:40:012
303039994233,5cyclictest337514-21cron09:05:012
303039994231,7cyclictest3615533-21cupsd12:01:182
303038994235,4cyclictest343122-21kworker/u8:11-xprtiod11:15:011
303038994233,6cyclictest3615537-21cups-browsed12:15:471
303038994231,7cyclictest433268-21ntpq11:05:391
30303799423,32cyclictest401055-21ssh10:25:370
303037994232,7cyclictest420505-21aten_r2power_fr10:50:250
303037994232,7cyclictest420505-21aten_r2power_fr10:50:250
303039994132,5cyclictest307549-21systemd-run07:25:012
303039994131,6cyclictest433637-21awk11:05:482
303039994131,6cyclictest303044-21kworker/u8:14+xprtiod07:10:012
30303999412,34cyclictest344312-21ssh09:15:382
303038994133,5cyclictest475122-21sendmail12:00:011
30303899413,33cyclictest457166-21ssh11:35:381
303038994132,6cyclictest395804-21cron10:20:011
303038994132,6cyclictest395804-21cron10:20:011
303038994131,4cyclictest0-21swapper/111:00:391
303038994130,7cyclictest365263-21unixbench-2d09:40:521
303038994130,7cyclictest365263-21unixbench-2d09:40:521
303037994132,2cyclictest0-21swapper/011:35:410
303039994034,3cyclictest0-21swapper/212:40:002
303039994031,5cyclictest467593-21latency_hist11:50:022
303039994031,5cyclictest411941-21idleruntime-cro10:40:012
303039994027,9cyclictest360745-21sshd09:35:412
30303999401,29cyclictest495101-21ssh12:25:372
30303999401,29cyclictest495101-21ssh12:25:372
303038994031,5cyclictest359405-21cron09:35:011
303037994031,6cyclictest420-21avahi-daemon09:36:060
303037994022,14cyclictest1-21systemd10:08:110
303039993931,5cyclictest3615537-21cups-browsed12:30:162
303039993931,5cyclictest3615537-21cups-browsed12:30:162
303039993931,5cyclictest339140-21latency_hist09:10:022
303039993930,4cyclictest70-21kswapd011:00:012
303039993929,6cyclictest351863-21aten_r2power_vo09:25:282
303039993927,9cyclictest400737-21kernelversion10:25:312
303038993932,4cyclictest444630-21aten_r2power_po11:20:241
303038993931,5cyclictest455944-21munin-plugin-st11:35:011
303038993931,5cyclictest455944-21munin-plugin-st11:35:011
303038993930,5cyclictest425777-21unixbench_multi10:55:491
303037993932,4cyclictest324639-21kworker/u8:1+xprtiod09:35:010
303037993930,6cyclictest3615537-21cups-browsed12:12:400
303037993930,6cyclictest261-21systemd-journal10:48:110
303037993930,6cyclictest261-21systemd-journal10:48:110
303037993918,18cyclictest12350irq/32-eth012:15:360
30303999384,24cyclictest364666-21ssh09:40:392
30303999384,24cyclictest364666-21ssh09:40:392
303039993829,4cyclictest0-21swapper/211:10:432
303038993829,5cyclictest310708-21latency_hist07:35:021
303038993828,7cyclictest413209-21ntp_offset10:40:401
303037993829,6cyclictest548-21gdbus11:46:030
303037993829,6cyclictest1051-21mta-sts-daemon08:44:110
303037993827,8cyclictest348449-21ntp_offset09:20:400
303037993827,8cyclictest348449-21ntp_offset09:20:400
303039993729,5cyclictest393646-21unixbench_singl10:15:522
303039993729,5cyclictest393646-21unixbench_singl10:15:522
303039993728,4cyclictest0-21swapper/210:10:392
303038993726,4cyclictest0-21swapper/109:55:381
30303899372,29cyclictest384829-21ssh10:05:351
303037993730,4cyclictest456123-21latency_hist11:35:030
303037993730,4cyclictest456123-21latency_hist11:35:030
303037993728,6cyclictest319548-21kworker/u8:2-xprtiod08:25:370
303037993728,5cyclictest420-21avahi-daemon09:15:300
303037993721,10cyclictest12350irq/32-eth010:40:380
30303999362,27cyclictest468764-21ssh11:50:422
303039993622,3cyclictest308512-21collect207:25:402
303038993630,3cyclictest0-21swapper/107:45:451
303038993629,3cyclictest0-21swapper/109:10:011
303038993626,6cyclictest420-21avahi-daemon07:25:421
303038993626,6cyclictest340781-21tail09:10:471
30303899362,28cyclictest449425-21ssh11:25:471
30303799364,9cyclictest171rcu_preempt09:55:390
303037993630,4cyclictest1055-21mta-sts-daemon10:20:340
303037993628,5cyclictest452013-21kworker/u8:0+xprtiod12:25:000
303037993628,5cyclictest452013-21kworker/u8:0+xprtiod12:25:000
303037993624,4cyclictest460820-21latency11:40:330
303037993620,14cyclictest547-21gmain07:50:360
303037993616,16cyclictest364868-21ssh09:40:470
303037993616,16cyclictest364868-21ssh09:40:470
30303999353,23cyclictest441249-21ssh11:15:362
303039993526,6cyclictest3615537-21cups-browsed10:45:512
303039993526,6cyclictest3615537-21cups-browsed10:45:512
303039993526,5cyclictest322541-21systemd-run08:15:012
303039993526,5cyclictest322541-21systemd-run08:15:012
303039993524,7cyclictest1051-21mta-sts-daemon08:00:372
303039993510,4cyclictest171rcu_preempt11:30:382
303039993510,4cyclictest171rcu_preempt11:30:382
30303899358,22cyclictest352445-21ssh09:25:371
303038993530,2cyclictest0-21swapper/109:20:011
303038993528,4cyclictest339015-21diskmemload12:05:361
303038993527,5cyclictest328400-21unixbench_multi08:30:481
30303799354,26cyclictest380951-21sshd10:00:440
303037993526,6cyclictest3615537-21cups-browsed10:14:060
30303799352,22cyclictest437238-21ssh11:10:370
303037993514,13cyclictest502660-21ssh12:35:420
30303799351,28cyclictest352471-21ssh09:25:390
303039993426,5cyclictest421815-21ssh10:50:482
303039993426,5cyclictest421815-21ssh10:50:482
303039993425,6cyclictest330016-21grep08:40:012
303039993425,5cyclictest1051-21mta-sts-daemon09:47:032
303039993425,5cyclictest1051-21mta-sts-daemon09:47:032
303039993424,7cyclictest461353-21mailstats11:40:482
303039993424,4cyclictest0-21swapper/210:33:442
303039993423,8cyclictest384301-21ssh10:05:282
303037993427,4cyclictest478949-21cut12:05:010
303037993426,5cyclictest3615537-21cups-browsed11:55:250
303037993424,7cyclictest372371-21sh09:50:310
30303799340,4cyclictest171rcu_preempt07:10:340
30303899333,23cyclictest495092-21sh12:25:371
30303899333,23cyclictest495092-21sh12:25:371
303038993326,4cyclictest1051-21mta-sts-daemon07:35:371
303038993325,5cyclictest371552-21ssh09:50:001
303038993325,5cyclictest371552-21ssh09:50:001
303038993324,6cyclictest3615537-21cups-browsed08:10:011
303037993324,6cyclictest445724-21timerwakeupswit11:20:440
303037993323,6cyclictest429206-21ssh11:00:360
303037993316,12cyclictest151ktimers/007:20:350
303037993310,16cyclictest340250-21rm09:10:370
303039993226,3cyclictest0-21swapper/207:45:012
303039993225,3cyclictest0-21swapper/209:10:462
303039993224,4cyclictest0-21swapper/212:20:332
303039993224,4cyclictest0-21swapper/212:20:332
303039993222,3cyclictest171rcu_preempt08:45:352
30303899323,22cyclictest400987-21ssh10:25:351
303038993224,5cyclictest476495-21sendmail_mailqu12:00:441
303038993224,5cyclictest318552-21fschecks_count08:00:281
303038993224,4cyclictest0-21swapper/109:35:351
303038993223,6cyclictest3615537-21cups-browsed11:44:241
30303799328,17cyclictest3615537-21cups-browsed11:25:400
30303799324,22cyclictest441-21gpm08:00:400
303037993225,4cyclictest337546-21systemd-run09:05:010
303039993125,3cyclictest0-21swapper/211:25:022
303039993123,5cyclictest330558-21fschecks_time08:40:282
303039993123,5cyclictest328508-21cron08:35:012
303039993123,5cyclictest318012-21munin-run08:00:012
303039993121,6cyclictest324062-21systemd-run08:20:012
303039993121,6cyclictest324062-21systemd-run08:20:012
303039993111,4cyclictest171rcu_preempt07:45:362
303038993125,3cyclictest0-21swapper/107:45:011
303038993124,4cyclictest695-21cron09:05:011
303038993123,5cyclictest316518-21cut07:55:001
303038993123,5cyclictest261-21systemd-journal09:00:011
303038993123,5cyclictest0-21swapper/110:35:011
303038993123,4cyclictest0-21swapper/111:45:251
303038993123,3cyclictest0-21swapper/108:00:011
303038993121,7cyclictest348973-21unixbench_multi09:20:491
303038993121,7cyclictest348973-21unixbench_multi09:20:491
303037993126,3cyclictest479568-21cpu12:05:250
303037993122,4cyclictest0-21swapper/011:50:400
303037993117,11cyclictest0-21swapper/008:00:000
303039993022,5cyclictest0-21swapper/207:20:012
303038993025,2cyclictest0-21swapper/108:40:011
303037993024,4cyclictest432619-21aten_r2power_po11:05:260
303039992916,3cyclictest171rcu_preempt08:50:362
30303999291,12cyclictest36-21ksoftirqd/207:10:402
303038992922,3cyclictest0-21swapper/110:12:481
303038992921,4cyclictest491991-21sed12:20:511
303038992921,4cyclictest491991-21sed12:20:511
303037992919,3cyclictest307537-21kworker/u8:4-xprtiod07:40:420
303039992820,4cyclictest0-21swapper/207:55:002
303038992822,3cyclictest0-21swapper/107:20:011
303038992819,6cyclictest3615537-21cups-browsed07:13:281
303037992820,6cyclictest337746-21sshd09:05:100
303037992819,6cyclictest333351-21aten_r2power_fr08:50:260
303037992819,6cyclictest12450irq/33-eth007:45:410
303039992718,6cyclictest547-21gmain07:30:242
303039992718,6cyclictest3615537-21cups-browsed08:59:552
30303899273,16cyclictest428-21mta-sts-daemon08:25:381
303038992719,5cyclictest332954-21unixbench_singl08:45:451
303037992720,4cyclictest407939-21latency_hist10:35:030
303037992719,4cyclictest324639-21kworker/u8:1-nfsiod08:55:350
30303999262,20cyclictest312019-21awk07:40:012
303039992618,4cyclictest1054-21mta-sts-daemon08:20:382
303039992618,4cyclictest1054-21mta-sts-daemon08:20:382
30303899264,17cyclictest659-21snmpd08:10:341
30303899264,17cyclictest659-21snmpd08:10:341
303038992618,5cyclictest324012-21cron08:20:001
303038992618,5cyclictest324012-21cron08:20:001
303038992618,5cyclictest307457-21unixbench_singl07:20:471
30303799262,14cyclictest151ktimers/008:15:370
30303799262,14cyclictest151ktimers/008:15:370
303039992518,4cyclictest319753-21kworker/u8:5-rpciod08:10:012
303039992518,3cyclictest0-21swapper/208:25:332
303037992520,3cyclictest151ktimers/007:25:390
303037992514,8cyclictest12450irq/33-eth008:45:330
30303799251,20cyclictest1-21systemd08:40:020
303038992313,4cyclictest326388-21kworker/u8:0-xprtiod08:50:381
303037992314,6cyclictest325437-21grep08:20:480
303037992314,6cyclictest325437-21grep08:20:480
30303799227,4cyclictest321354-21aten_r2power_fr08:10:250
30303799227,4cyclictest321354-21aten_r2power_fr08:10:250
30303799227,12cyclictest312019-21idleruntime-cro07:40:010
303038992113,4cyclictest1054-21mta-sts-daemon08:20:391
303038992113,4cyclictest1054-21mta-sts-daemon08:20:391
303037992116,2cyclictest0-21swapper/008:05:290
30304099202,8cyclictest0-21swapper/310:00:383
30303799209,8cyclictest303177-21wc07:10:020
30304099182,9cyclictest0-21swapper/311:45:413
30304099182,9cyclictest0-21swapper/311:25:403
30304099181,9cyclictest0-21swapper/311:50:403
30303899187,8cyclictest303122-21latency_hist07:10:021
30304099173,9cyclictest0-21swapper/309:30:383
30304099173,6cyclictest0-21swapper/311:35:303
30304099172,9cyclictest0-21swapper/311:15:373
30304099172,6cyclictest0-21swapper/311:20:323
30304099171,9cyclictest0-21swapper/312:35:353
30304099171,7cyclictest0-21swapper/307:10:353
30304099171,6cyclictest0-21swapper/307:30:343
30304099164,6cyclictest0-21swapper/310:15:473
30304099164,6cyclictest0-21swapper/310:15:473
30304099163,8cyclictest0-21swapper/311:05:363
30304099163,7cyclictest0-21swapper/312:20:443
30304099163,7cyclictest0-21swapper/312:20:443
30304099163,6cyclictest0-21swapper/310:05:373
30304099162,8cyclictest0-21swapper/310:40:373
30304099162,8cyclictest0-21swapper/309:10:353
30304099161,9cyclictest0-21swapper/312:00:383
30304099161,9cyclictest0-21swapper/308:25:343
30304099161,8cyclictest0-21swapper/312:25:403
30304099161,8cyclictest0-21swapper/312:25:403
30304099161,8cyclictest0-21swapper/312:05:383
30304099161,6cyclictest0-21swapper/308:35:343
30304099161,6cyclictest0-21swapper/307:40:393
30304099155,6cyclictest0-21swapper/311:10:413
30304099152,7cyclictest0-21swapper/311:30:283
30304099152,7cyclictest0-21swapper/311:30:283
30304099152,7cyclictest0-21swapper/310:45:363
30304099152,7cyclictest0-21swapper/310:45:363
30304099152,6cyclictest0-21swapper/308:40:343
30304099152,5cyclictest0-21swapper/308:55:363
30304099152,5cyclictest0-21swapper/307:45:363
30304099151,8cyclictest0-21swapper/310:35:383
30304099151,8cyclictest0-21swapper/310:25:383
30304099151,8cyclictest0-21swapper/309:50:353
30304099151,7cyclictest0-21swapper/309:40:413
30304099151,7cyclictest0-21swapper/309:40:413
30304099151,7cyclictest0-21swapper/308:00:393
30304099151,6cyclictest0-21swapper/310:55:323
30304099151,6cyclictest0-21swapper/308:05:353
30304099151,6cyclictest0-21swapper/307:55:363
30304099151,6cyclictest0-21swapper/307:50:353
30304099151,6cyclictest0-21swapper/307:25:333
30304099151,5cyclictest0-21swapper/308:45:323
30304099143,8cyclictest0-21swapper/309:55:313
30304099143,8cyclictest0-21swapper/309:45:263
30304099143,8cyclictest0-21swapper/309:45:263
30304099143,6cyclictest0-21swapper/311:00:393
30304099142,7cyclictest0-21swapper/311:55:283
30304099142,6cyclictest0-21swapper/309:25:283
30304099141,9cyclictest0-21swapper/312:30:363
30304099141,9cyclictest0-21swapper/312:30:363
30304099141,8cyclictest0-21swapper/312:15:353
30304099141,8cyclictest0-21swapper/310:30:363
30304099141,7cyclictest0-21swapper/312:10:343
30304099141,7cyclictest0-21swapper/308:20:353
30304099141,7cyclictest0-21swapper/308:20:353
30304099141,6cyclictest0-21swapper/307:15:403
30304099133,5cyclictest0-21swapper/307:20:383
30304099132,7cyclictest0-21swapper/309:15:363
30304099132,6cyclictest0-21swapper/310:50:423
30304099132,6cyclictest0-21swapper/310:50:423
30304099132,6cyclictest0-21swapper/310:20:253
30304099132,6cyclictest0-21swapper/309:35:263
30304099131,8cyclictest0-21swapper/311:40:383
30304099131,7cyclictest0-21swapper/310:10:363
30304099131,6cyclictest0-21swapper/309:20:263
30304099131,6cyclictest0-21swapper/309:20:263
30304099131,6cyclictest0-21swapper/309:05:363
30304099131,6cyclictest0-21swapper/308:20:013
30304099131,6cyclictest0-21swapper/308:20:013
30304099131,6cyclictest0-21swapper/308:10:363
30304099131,6cyclictest0-21swapper/308:10:363
30304099121,6cyclictest0-21swapper/309:00:333
30304099121,5cyclictest0-21swapper/308:30:363
30304099111,6cyclictest0-21swapper/307:35:343
30304099102,5cyclictest0-21swapper/308:50:343
3030409981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional