You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 23:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Jan 01, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2760899260,25cyclictest0-21swapper/211:05:542
2760899260,1cyclictest32372-21sendmail-msp07:20:012
2761299240,23cyclictest1310-21awk08:30:013
27606992421,2cyclictest16885-21cron07:55:001
2760099240,2cyclictest4180-21awk10:45:010
27612992321,1cyclictest27514-21latency_hist12:40:013
2761299230,22cyclictest12986-21sendmail10:00:013
2761299230,22cyclictest12986-21sendmail10:00:013
2760899230,22cyclictest5839-21latency_hist11:55:012
27606992323,0cyclictest0-21swapper/112:25:561
27606992320,2cyclictest22987-21idleruntime-cro10:20:001
27606992320,2cyclictest12100-21sh07:45:011
2760699230,2cyclictest16640-21grep11:10:001
2760699230,22cyclictest5824-21latency_hist11:55:011
2760699230,22cyclictest4907-21apt-get10:45:551
2760699230,22cyclictest0-21swapper/110:25:021
2760099230,22cyclictest31044-21latency_hist11:40:010
27612992222,0cyclictest0-21swapper/310:30:023
27612992220,1cyclictest3346-21latency_hist09:40:003
2761299220,21cyclictest9751-21apt-get10:55:533
2761299220,21cyclictest4424-21apt-get08:35:533
2761299220,21cyclictest21708-21dump-pmu-power08:05:003
2761299220,21cyclictest0-21swapper/312:15:013
2761299220,21cyclictest0-21swapper/312:00:533
2761299220,1cyclictest0-21swapper/308:15:013
27608992220,1cyclictest24429-21apt-get08:10:532
27608992220,1cyclictest11375-21munin-run08:50:002
2760899220,21cyclictest18534-21apt-get10:10:532
2760899220,21cyclictest0-21swapper/209:15:552
2760899220,21cyclictest0-21swapper/209:15:012
2760899220,1cyclictest606-21dbus-daemon10:20:002
27606992221,0cyclictest0-21swapper/109:10:531
27606992220,1cyclictest605-21avahi-daemon08:20:011
27606992219,2cyclictest29935-21cron07:15:001
2760699220,21cyclictest0-21swapper/111:30:531
2760699220,21cyclictest0-21swapper/110:15:011
2760699220,1cyclictest27484-21turbostat.cron12:40:011
27600992222,0cyclictest25414-21apt-get12:35:520
27600992221,1cyclictest1029-21ls11:45:010
27600992220,1cyclictest9402-21cron10:55:010
27600992220,1cyclictest29941-21perf07:15:000
27600992220,1cyclictest28646-21munin-run11:35:010
27600992220,1cyclictest13099-21perf12:10:000
27600992219,2cyclictest16913-21munin-plugin-st07:55:000
2760099220,21cyclictest1353-21apt-get11:45:550
2760099220,21cyclictest0-21swapper/009:50:520
27612992121,0cyclictest17221-21apt-get07:55:523
27612992120,1cyclictest19282-21apt-get09:05:533
27612992120,1cyclictest19282-21apt-get09:05:533
27612992120,1cyclictest16886-21apt-get09:00:503
27612992120,1cyclictest0-21swapper/307:15:553
2761299210,21cyclictest0-21swapper/309:40:533
2761299210,21cyclictest0-21swapper/307:10:533
2761299210,20cyclictest28934-21sendmail-msp08:20:013
2761299210,20cyclictest12432-21apt-get07:45:523
2761299210,20cyclictest0-21swapper/308:05:513
2761299210,1cyclictest13087-21latency_hist12:10:003
2761299210,1cyclictest0-21swapper/309:30:523
2761299210,0cyclictest32688-21apt-get07:20:533
2761299210,0cyclictest28896-21apt-get09:25:523
2761299210,0cyclictest13351-21apt-get10:00:533
2761299210,0cyclictest0-21swapper/307:30:523
27608992120,1cyclictest0-21swapper/211:15:532
27608992120,0cyclictest0-21swapper/210:00:532
27608992117,4cyclictest0-21swapper/211:10:532
2760899210,20cyclictest31719-21apt-get10:35:542
2760899210,20cyclictest26527-21apt-get10:25:532
2760899210,20cyclictest15810-21apt-get12:15:542
2760899210,20cyclictest10589-21cron09:55:012
2760899210,20cyclictest0-21swapper/211:45:552
2760899210,20cyclictest0-21swapper/208:50:522
2760899210,1cyclictest313-21systemd-journal09:40:002
2760899210,1cyclictest17220-21apt07:55:522
2760899210,1cyclictest0-21swapper/207:10:532
2760899210,0cyclictest0-21swapper/212:35:522
27606992121,0cyclictest0-21swapper/110:30:021
27606992120,1cyclictest0-21swapper/110:05:561
27606992119,1cyclictest17854-21cron12:20:011
27606992119,1cyclictest0-21swapper/107:55:521
2760699210,20cyclictest0-21swapper/109:00:511
2760699210,0cyclictest2670-21apt-get07:25:511
2760699210,0cyclictest2670-21apt-get07:25:511
2760699210,0cyclictest0-21swapper/110:40:021
2760699210,0cyclictest0-21swapper/109:40:521
27600992121,0cyclictest0-21swapper/011:50:530
27600992121,0cyclictest0-21swapper/009:55:530
27600992121,0cyclictest0-21swapper/009:55:530
27600992121,0cyclictest0-21swapper/008:00:530
27600992119,2cyclictest20929-21apt-get10:15:540
27600992119,1cyclictest605-21avahi-daemon07:40:010
27600992119,1cyclictest19305-21wc08:00:000
2760099210,21cyclictest0-21swapper/012:00:530
2760099210,20cyclictest11682-21apt-get08:50:510
2760099210,20cyclictest0-21swapper/010:10:530
2760099210,1cyclictest0-21swapper/010:35:540
2760099210,0cyclictest6156-21apt-get11:55:550
2760099210,0cyclictest26824-21apt-get08:15:540
2760099210,0cyclictest19350-21apt-get11:15:530
2760099210,0cyclictest0-21swapper/012:30:530
2760099210,0cyclictest0-21swapper/008:55:520
2760099210,0cyclictest0-21swapper/007:40:500
27612992020,0cyclictest0-21swapper/310:40:023
27612992020,0cyclictest0-21swapper/308:45:533
27612992020,0cyclictest0-21swapper/307:50:513
27612992020,0cyclictest0-21swapper/307:25:513
27612992020,0cyclictest0-21swapper/307:25:513
27612992019,1cyclictest21771-21apt-get11:20:543
27612992018,1cyclictest3759-21apt-get11:50:533
27612992018,1cyclictest31377-21apt-get11:40:543
27612992018,1cyclictest26174-21perf09:20:013
27612992018,1cyclictest0-21swapper/308:45:003
2761299200,0cyclictest24164-21apt-get11:25:553
2761299200,0cyclictest16956-21apt-get11:10:543
2761299200,0cyclictest0-21swapper/310:35:543
2761299200,0cyclictest0-21swapper/308:50:523
27608992020,0cyclictest7515-21apt-get07:35:512
27608992020,0cyclictest20627-21apt-get12:25:562
27608992020,0cyclictest0-21swapper/212:10:532
27608992020,0cyclictest0-21swapper/212:05:532
27608992020,0cyclictest0-21swapper/210:50:532
27608992020,0cyclictest0-21swapper/210:45:552
27608992020,0cyclictest0-21swapper/210:40:022
27608992020,0cyclictest0-21swapper/209:25:522
27608992019,1cyclictest26559-21apt-get11:30:532
27608992019,1cyclictest1624-21apt-get08:30:532
27608992018,1cyclictest0-21swapper/209:20:532
2760899200,20cyclictest0-21swapper/211:40:542
2760899200,0cyclictest0-21swapper/208:00:532
27606992020,0cyclictest13415-21apt-get12:10:541
27606992020,0cyclictest0-21swapper/112:30:521
27606992020,0cyclictest0-21swapper/110:50:531
27606992020,0cyclictest0-21swapper/108:40:531
27606992020,0cyclictest0-21swapper/108:25:521
27606992020,0cyclictest0-21swapper/108:10:531
27606992018,1cyclictest1368-21apt-get10:40:011
27606992017,2cyclictest26186-21awk09:20:011
2760699200,19cyclictest0-21swapper/109:55:011
2760699200,0cyclictest0-21swapper/109:05:531
2760699200,0cyclictest0-21swapper/109:05:531
2760699200,0cyclictest0-21swapper/108:20:541
2760699200,0cyclictest0-21swapper/107:35:511
27600992020,0cyclictest9288-21apt-get08:45:530
27600992020,0cyclictest0-21swapper/012:15:540
27600992020,0cyclictest0-21swapper/012:10:530
27600992020,0cyclictest0-21swapper/011:20:540
27600992020,0cyclictest0-21swapper/009:25:530
27600992020,0cyclictest0-21swapper/008:30:530
27600992020,0cyclictest0-21swapper/007:15:550
27600992019,1cyclictest0-21swapper/011:00:520
27600992019,1cyclictest0-21swapper/009:45:550
27600992018,1cyclictest957-21perf09:35:000
2760099200,0cyclictest6840-21apt-get08:40:530
2760099200,0cyclictest0-21swapper/007:45:530
27612991919,0cyclictest0-21swapper/312:15:543
27612991919,0cyclictest0-21swapper/311:35:563
27612991919,0cyclictest0-21swapper/310:45:553
27612991919,0cyclictest0-21swapper/307:40:493
27612991919,0cyclictest0-21swapper/307:35:513
27612991918,0cyclictest26495-21apt-get09:20:533
27612991917,1cyclictest16631-21dump-pmu-power11:10:003
27608991919,0cyclictest0-21swapper/210:20:522
27608991919,0cyclictest0-21swapper/209:55:522
27608991919,0cyclictest0-21swapper/209:55:522
27608991917,1cyclictest14236-21perf11:05:002
27608991917,1cyclictest0-21swapper/209:00:502
2760899190,1cyclictest0-21swapper/212:01:022
2760899190,0cyclictest0-21swapper/210:30:552
27606991919,0cyclictest20297-21perf12:25:011
27606991919,0cyclictest13351-21apt-get10:00:521
27606991918,0cyclictest23742-21apt-get10:20:521
27606991917,1cyclictest3386-21sendmail-msp09:40:001
2760699190,18cyclictest19012-21basename11:15:001
27600991919,0cyclictest0-21swapper/011:25:540
27600991919,0cyclictest0-21swapper/010:25:530
27600991919,0cyclictest0-21swapper/010:00:530
27600991919,0cyclictest0-21swapper/007:30:520
27612991818,0cyclictest0-21swapper/311:15:533
27612991818,0cyclictest0-21swapper/310:05:563
27612991818,0cyclictest0-21swapper/308:20:543
27612991817,0cyclictest0-21swapper/310:20:523
27608991818,0cyclictest6093-21apt-get09:45:542
27608991818,0cyclictest0-21swapper/208:25:522
27608991818,0cyclictest0-21swapper/207:20:532
27608991813,1cyclictest7263-21diskstats08:40:542
2760899181,1cyclictest0-21swapper/208:35:022
27606991818,0cyclictest0-21swapper/111:20:541
27606991818,0cyclictest0-21swapper/108:30:521
27600991818,0cyclictest29250-21apt-get08:20:540
27600991818,0cyclictest0-21swapper/008:35:530
27612991715,1cyclictest16177-21munin-run09:00:013
2761299170,16cyclictest34-21ksoftirqd/310:25:023
27608991717,0cyclictest0-21swapper/207:40:502
27608991715,1cyclictest16122-21cron09:00:012
27608991715,1cyclictest0-21swapper/209:35:002
27606991717,0cyclictest0-21swapper/111:45:551
27606991717,0cyclictest0-21swapper/111:15:531
27606991717,0cyclictest0-21swapper/109:45:551
27606991717,0cyclictest0-21swapper/109:35:001
27600991717,0cyclictest0-21swapper/012:20:550
27600991717,0cyclictest0-21swapper/011:10:540
27600991717,0cyclictest0-21swapper/010:20:520
27600991717,0cyclictest0-21swapper/010:05:560
27600991715,1cyclictest6987-21latency_hist10:50:010
2760099170,16cyclictest11820-21sendmail11:00:010
27612991616,0cyclictest0-21swapper/311:30:533
27612991616,0cyclictest0-21swapper/311:00:533
27612991616,0cyclictest0-21swapper/309:50:513
27608991613,2cyclictest23849-21grep11:25:002
2760899160,14cyclictest0-21swapper/211:56:252
2760899160,14cyclictest0-21swapper/209:05:302
2760899160,14cyclictest0-21swapper/209:05:302
2760899160,14cyclictest0-21swapper/208:16:122
2760899160,14cyclictest0-21swapper/207:54:382
2760899160,14cyclictest0-21swapper/207:46:562
27606991614,1cyclictest30617-21users09:26:031
2760699160,0cyclictest0-21swapper/109:55:531
2760699160,0cyclictest0-21swapper/109:55:531
27600991616,0cyclictest0-21swapper/007:20:520
27600991615,0cyclictest22031-21apt-get08:05:510
27600991614,1cyclictest28567-21latency_hist09:25:010
27612991515,0cyclictest0-21swapper/312:25:563
27612991515,0cyclictest0-21swapper/311:55:553
27612991515,0cyclictest0-21swapper/310:15:543
27612991515,0cyclictest0-21swapper/308:30:523
2761299150,14cyclictest8220-21apt09:50:013
27608991513,1cyclictest23635-21tail12:30:552
27606991515,0cyclictest26495-21apt-get09:20:531
27606991515,0cyclictest0-21swapper/108:55:521
2760699150,1cyclictest9237-21head12:00:561
2760699150,1cyclictest20689-21seq08:00:591
2760699150,1cyclictest13232-21irqstats07:45:561
27600991515,0cyclictest0-21swapper/009:35:540
2760099150,14cyclictest20187-21memory09:05:570
2760099150,14cyclictest20187-21memory09:05:570
27612991414,0cyclictest0-21swapper/310:50:293
27612991414,0cyclictest0-21swapper/309:10:303
2761299140,0cyclictest0-21swapper/312:30:523
2761299140,0cyclictest0-21swapper/312:21:393
2761299140,0cyclictest0-21swapper/311:45:033
2761299140,0cyclictest0-21swapper/310:11:253
27608991414,0cyclictest0-21swapper/212:21:172
27608991414,0cyclictest0-21swapper/207:30:022
2760899140,14cyclictest0-21swapper/210:05:282
2760899140,14cyclictest0-21swapper/208:20:092
2760899140,14cyclictest0-21swapper/207:25:112
2760899140,14cyclictest0-21swapper/207:25:112
2760899140,0cyclictest0-21swapper/211:35:332
2760899140,0cyclictest0-21swapper/211:25:592
2760899140,0cyclictest0-21swapper/209:41:002
2760899140,0cyclictest0-21swapper/208:05:262
27606991414,0cyclictest0-21swapper/112:05:141
27606991414,0cyclictest0-21swapper/111:55:051
27606991414,0cyclictest0-21swapper/108:45:541
27606991414,0cyclictest0-21swapper/107:30:021
2760699140,14cyclictest0-21swapper/111:35:441
2760699140,14cyclictest0-21swapper/111:25:131
2760699140,14cyclictest0-21swapper/107:20:201
2760699140,14cyclictest0-21swapper/107:15:041
2760699140,0cyclictest0-21swapper/108:06:151
27600991414,0cyclictest0-21swapper/012:25:400
27600991414,0cyclictest0-21swapper/009:11:170
27600991414,0cyclictest0-21swapper/007:25:340
27600991414,0cyclictest0-21swapper/007:25:340
2760099140,14cyclictest0-21swapper/010:30:090
2760099140,14cyclictest0-21swapper/009:15:110
2760099140,14cyclictest0-21swapper/009:01:210
2760099140,14cyclictest0-21swapper/008:25:210
2760099140,0cyclictest0-21swapper/011:08:000
2760099140,0cyclictest0-21swapper/009:40:330
2760099140,0cyclictest0-21swapper/008:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional