You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 14:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Tue Sep 08, 2026 12:42:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
965928996047,10cyclictest12650irq/32-eth009:51:090
96593099513,39cyclictest1045952-21ssh10:21:102
965928994940,6cyclictest421-21dbus-daemon09:46:110
965928994927,18cyclictest0-21swapper/010:50:020
965928994821,3cyclictest0-21swapper/009:35:010
965929994739,5cyclictest982445-21systemd-run08:05:021
965928994723,21cyclictest1092078-21unixbench_multi11:36:220
965928994635,8cyclictest12650irq/32-eth010:21:080
965928994632,7cyclictest12650irq/32-eth010:06:070
965928994626,17cyclictest186014-21cups-browsed10:32:050
965928994620,2cyclictest0-21swapper/007:20:010
965930994536,5cyclictest0-21swapper/208:00:012
96593099453,33cyclictest1058086-21ssh10:41:092
965929994519,13cyclictest1096972-21aten_r2power_fr11:45:591
965928994522,20cyclictest420-21avahi-daemon09:21:030
965930994434,6cyclictest1091471-21ntp_kernel_err11:36:102
965928994422,3cyclictest1056874-21systemd10:40:020
965928994422,3cyclictest1056874-21systemd10:40:020
965928994418,3cyclictest0-21swapper/011:41:050
965929994337,3cyclictest0-21swapper/111:16:061
965929994336,4cyclictest1106639-21needreboot12:01:061
965928994335,3cyclictest0-21swapper/007:26:100
965928994323,4cyclictest0-21swapper/012:31:100
96592899431,20cyclictest1097584-21sh11:46:090
965930994233,5cyclictest1032488-21cron10:00:012
965930994233,5cyclictest1032488-21cron10:00:012
965930994233,5cyclictest1021172-21fschecks_count09:41:022
965929994212,12cyclictest1018675-21ps09:36:171
96592899422,7cyclictest421-21dbus-daemon10:13:450
965930994132,5cyclictest421-21dbus-daemon12:06:172
965929994133,5cyclictest1-21systemd09:20:011
965929994132,6cyclictest186014-21cups-browsed09:31:071
965929994112,8cyclictest1021587-21ntpq09:41:101
965928994134,3cyclictest0-21swapper/011:06:060
965928994126,3cyclictest1082464-21ssh11:21:140
965928994126,3cyclictest1082464-21ssh11:21:140
965928994126,12cyclictest1050738-21cron10:30:010
965928994117,2cyclictest0-21swapper/009:36:170
965928994117,2cyclictest0-21swapper/007:12:040
965930994031,6cyclictest985432-21systemd-run08:15:012
965930994030,6cyclictest186014-21cups-browsed12:00:062
965930994027,9cyclictest420-21avahi-daemon11:16:172
965929994034,4cyclictest1002-21mta-sts-daemon09:46:001
965929994017,7cyclictest978762-21ntpq07:51:051
965929994011,10cyclictest1112797-21ntpq12:11:131
965929994011,10cyclictest1112797-21ntpq12:11:131
965928994034,4cyclictest1117654-21ssh12:20:010
96592899403,28cyclictest420-21avahi-daemon09:11:090
96592899403,28cyclictest420-21avahi-daemon09:11:090
96592899402,23cyclictest1128110-21ssh12:36:140
965928994015,4cyclictest0-21swapper/010:40:420
965928994013,7cyclictest12650irq/32-eth011:31:140
96592899401,19cyclictest1064283-21ssh10:51:140
965930993933,3cyclictest0-21swapper/210:40:012
965930993933,3cyclictest0-21swapper/210:40:012
96592999399,8cyclictest1125789-21ssh12:32:001
965929993934,2cyclictest0-21swapper/109:21:031
965929993933,3cyclictest0-21swapper/110:01:141
965929993932,4cyclictest1003-21mta-sts-daemon10:53:451
965929993914,8cyclictest996822-21ntpq08:51:091
965929993912,8cyclictest995376-21ntp_states08:46:121
965929993912,10cyclictest992559-21switchtime08:36:201
965928993931,5cyclictest12650irq/32-eth012:21:050
96592899393,10cyclictest1042872-21ssh10:16:120
965928993929,7cyclictest12650irq/32-eth011:26:090
965928993929,7cyclictest12650irq/32-eth011:26:090
965928993914,7cyclictest994954-21fschecks_time08:46:020
965930993830,4cyclictest0-21swapper/209:11:022
965930993830,4cyclictest0-21swapper/209:11:022
965930993829,6cyclictest1000413-21cron09:05:002
965930993828,6cyclictest997395-21cron08:55:002
965930993827,8cyclictest1011665-21acpi09:25:572
965929993832,3cyclictest0-21swapper/107:20:011
965929993831,3cyclictest0-21swapper/108:56:051
965929993824,8cyclictest993846-21ntpq08:41:121
965929993824,8cyclictest993846-21ntpq08:41:121
965929993811,8cyclictest1071943-21ssh11:04:371
96592899383,15cyclictest665-21snmpd10:01:090
965928993830,5cyclictest982439-21systemd-run08:05:010
965928993830,5cyclictest1070801-21unixbench-2d11:01:220
96592899382,31cyclictest1106773-21ssh12:01:110
965930993731,4cyclictest1097339-21kernelversion11:46:042
965930993730,4cyclictest0-21swapper/209:20:012
965930993729,5cyclictest992799-21tail08:36:212
965930993729,5cyclictest1009019-21ssh09:21:042
965930993729,3cyclictest0-21swapper/209:36:102
965929993731,3cyclictest0-21swapper/111:11:101
965929993731,3cyclictest0-21swapper/110:41:161
965929993729,5cyclictest1084338-21latency_hist11:25:021
965929993729,5cyclictest1084338-21latency_hist11:25:021
96592999372,31cyclictest1049034-21rm10:26:111
96592999372,28cyclictest1036749-21ssh10:06:071
965929993711,8cyclictest981026-21wc08:00:021
96592899379,3cyclictest171rcu_preempt11:16:130
965930993629,4cyclictest998910-21awk09:00:012
965930993628,3cyclictest0-21swapper/211:01:142
965930993624,9cyclictest1066955-21kernelversion10:56:042
96592999369,10cyclictest1041261-21ssh10:13:571
965929993630,3cyclictest0-21swapper/112:00:011
965929993625,8cyclictest1073168-21missed_timers11:06:081
965929993610,8cyclictest1042967-21ntpq10:16:161
965928993629,4cyclictest1123908-21latency_hist12:30:020
965928993629,3cyclictest0-21swapper/011:51:040
965930993527,5cyclictest1085264-21sh11:26:052
965930993527,5cyclictest1085264-21sh11:26:052
965930993524,7cyclictest1100483-21missed_timers11:51:042
96593099352,22cyclictest1036820-21ssh10:06:102
96592999358,21cyclictest186014-21cups-browsed09:26:111
96592999353,25cyclictest1027603-21ssh09:51:081
965929993528,3cyclictest0-21swapper/109:06:121
965929993527,5cyclictest1109367-21fschecks_time12:06:021
965929993527,4cyclictest0-21swapper/111:26:021
965929993527,4cyclictest0-21swapper/111:26:021
965929993526,5cyclictest969786-21ntpq07:21:101
965929993524,5cyclictest999822-21ntpq09:01:101
965929993514,4cyclictest1001913-21diskmemload12:26:261
965929993514,4cyclictest1001913-21diskmemload09:14:571
965929993514,4cyclictest1001913-21diskmemload09:14:571
965929993512,6cyclictest1001913-21diskmemload10:20:291
965928993526,7cyclictest1023051-21sshd09:44:040
965928993520,12cyclictest980952-21latency_hist08:00:020
965930993427,4cyclictest1105610-21latency_hist12:00:022
965930993426,5cyclictest0-21swapper/208:45:002
965930993426,5cyclictest0-21swapper/208:45:002
965930993426,4cyclictest0-21swapper/210:16:162
96592999347,8cyclictest1101695-21ssh11:52:421
965929993427,4cyclictest0-21swapper/108:35:011
965929993426,5cyclictest976955-21fschecks_time07:46:011
965929993426,5cyclictest1003-21mta-sts-daemon11:41:401
965929993425,6cyclictest976418-21munin-run07:45:001
965929993423,8cyclictest986324-21ntpq08:16:091
965929993423,7cyclictest1061181-21ntpq10:46:111
965929993421,8cyclictest1055130-21ntpq10:36:151
965929993421,8cyclictest1055130-21ntpq10:36:151
965929993410,7cyclictest988995-21awk08:26:031
965929993410,7cyclictest1001913-21diskmemload09:56:161
965929993410,7cyclictest1001913-21diskmemload09:56:161
965928993428,4cyclictest970431-21idleruntime-cro07:25:010
965928993425,6cyclictest265-21systemd-journal08:41:040
965928993425,6cyclictest265-21systemd-journal08:41:040
965928993424,7cyclictest186014-21cups-browsed08:26:010
96592899342,26cyclictest1103661-21ssh11:56:080
96593099334,22cyclictest186014-21cups-browsed09:06:122
965930993326,4cyclictest1115440-21gltestperf12:16:022
965930993325,5cyclictest968907-21cron07:20:012
965930993325,4cyclictest1002-21mta-sts-daemon09:51:082
965930993324,7cyclictest1112631-21sshd12:11:092
965930993324,7cyclictest1112631-21sshd12:11:092
965930993324,6cyclictest991429-21true08:35:012
965930993324,5cyclictest0-21swapper/210:11:112
965930993322,8cyclictest1073172-21missed_timers11:06:082
96593099331,25cyclictest1052026-21ssh10:31:132
96592999339,6cyclictest1001913-21diskmemload10:59:581
965929993322,7cyclictest1115784-21ntpq12:16:081
965929993321,8cyclictest987901-21ntpq08:21:141
965929993321,8cyclictest971344-21ntpq07:26:101
965929993321,7cyclictest984808-21ntpq08:11:101
965929993321,7cyclictest983409-21ntpq08:06:171
965929993310,7cyclictest1120557-21ssh12:24:401
96592899333,22cyclictest1001913-21diskmemload09:56:110
96592899333,22cyclictest1001913-21diskmemload09:56:110
965928993328,3cyclictest1008048-21munin-run09:20:000
965928993324,6cyclictest420-21avahi-daemon09:01:020
965928993323,8cyclictest1000-21mta-sts-daemon08:15:010
965928993317,13cyclictest151ktimers/008:06:120
96592899331,23cyclictest665-21snmpd07:31:100
96592899331,23cyclictest665-21snmpd07:31:100
96593099323,21cyclictest1015458-21ssh09:31:112
965930993226,3cyclictest0-21swapper/208:20:012
965930993225,3cyclictest0-21swapper/212:36:162
965930993222,6cyclictest976416-21cron07:45:002
965930993219,3cyclictest171rcu_preempt12:21:072
965930993219,2cyclictest171rcu_preempt07:11:092
965930993218,3cyclictest171rcu_preempt10:01:202
965930993214,6cyclictest1124391-21ssh12:30:592
965929993225,4cyclictest972952-21processes07:31:181
965929993225,4cyclictest972952-21processes07:31:181
965929993224,4cyclictest1001-21mta-sts-daemon10:30:191
965929993223,7cyclictest1090976-21sshd11:36:011
965929993219,8cyclictest974334-21ntpq07:36:091
965929993218,9cyclictest966786-21ntpq07:11:071
965928993224,5cyclictest1078163-21systemd-run11:15:020
965928993224,5cyclictest1066680-21aten2.4-expect10:55:590
965928993222,5cyclictest151ktimers/007:41:140
96592899322,23cyclictest421-21dbus-daemon08:21:150
965930993126,3cyclictest0-21swapper/211:35:012
965930993123,5cyclictest1062999-21latency_hist10:50:022
965930993122,6cyclictest1121697-21latency12:26:042
965930993120,8cyclictest989274-21ntp_kernel_err08:26:082
965930993119,6cyclictest171rcu_preempt10:51:202
965930993113,14cyclictest421-21dbus-daemon08:46:102
965929993122,6cyclictest539-21wpa_supplicant11:31:001
965928993124,3cyclictest0-21swapper/012:11:240
965928993124,3cyclictest0-21swapper/012:11:240
96593099308,3cyclictest171rcu_preempt07:46:132
96593099307,9cyclictest1047945-21sshd10:25:042
96593099306,10cyclictest341rcuc/207:24:302
965930993023,3cyclictest0-21swapper/211:41:092
965930993020,3cyclictest171rcu_preempt09:46:112
965930993015,3cyclictest171rcu_preempt07:55:022
965930992922,4cyclictest982301-21unixbench_multi08:01:202
965930992922,4cyclictest0-21swapper/207:40:002
965930992914,3cyclictest171rcu_preempt08:21:142
96593099290,11cyclictest972887-21ntpq07:31:152
96593099290,11cyclictest972887-21ntpq07:31:152
965929992914,11cyclictest1128741-21ssh12:36:421
965928992923,4cyclictest997927-21sort08:55:590
965928992922,5cyclictest420-21avahi-daemon07:51:070
965928992915,11cyclictest265-21systemd-journal08:40:010
965928992915,11cyclictest1000957-21fschecks_time09:06:030
965930992818,3cyclictest171rcu_preempt11:11:362
965930992817,3cyclictest171rcu_preempt11:21:142
965930992817,3cyclictest171rcu_preempt11:21:142
965928992824,2cyclictest0-21swapper/007:50:010
96592899282,19cyclictest1012395-21ssh09:26:100
96593099277,14cyclictest421-21dbus-daemon08:06:102
965930992713,3cyclictest0-21swapper/207:26:142
965929992717,7cyclictest965942-21munin-run07:10:021
965928992720,5cyclictest958-21sendmail-mta12:05:190
965928992720,4cyclictest965941-21systemd-run07:10:010
965928992718,6cyclictest996723-21needreboot08:51:050
965930992615,3cyclictest171rcu_preempt07:10:032
965928992412,10cyclictest0-21swapper/008:16:000
965928992313,7cyclictest750-21ntpd07:36:080
96592899231,10cyclictest991350-21unixbench_singl08:31:190
96593199213,9cyclictest0-21swapper/310:16:143
96593199192,10cyclictest0-21swapper/308:21:153
96593199183,7cyclictest0-21swapper/312:31:083
96593199182,8cyclictest0-21swapper/310:06:143
96593199182,8cyclictest0-21swapper/309:46:083
96593199173,7cyclictest0-21swapper/309:21:123
96593199172,8cyclictest0-21swapper/311:01:153
96593199171,9cyclictest0-21swapper/310:56:103
96593199171,9cyclictest0-21swapper/310:01:103
96593199171,7cyclictest0-21swapper/308:01:083
96593199163,8cyclictest0-21swapper/309:11:093
96593199163,8cyclictest0-21swapper/309:11:093
96593199162,9cyclictest0-21swapper/309:36:123
96593199162,8cyclictest0-21swapper/310:35:563
96593199162,8cyclictest0-21swapper/310:35:563
96593199162,8cyclictest0-21swapper/309:51:083
96593199162,8cyclictest0-21swapper/309:16:093
96593199161,9cyclictest0-21swapper/311:36:133
96593199152,8cyclictest0-21swapper/310:26:083
96593199152,8cyclictest0-21swapper/309:56:123
96593199152,8cyclictest0-21swapper/309:56:123
96593199152,7cyclictest0-21swapper/311:20:583
96593199152,7cyclictest0-21swapper/311:20:583
96593199151,9cyclictest0-21swapper/311:31:143
96593199151,9cyclictest0-21swapper/310:48:243
96593199151,9cyclictest0-21swapper/308:35:013
96593199151,8cyclictest0-21swapper/312:06:103
96593199151,8cyclictest0-21swapper/310:11:033
96593199151,7cyclictest0-21swapper/309:41:083
96593199151,7cyclictest0-21swapper/307:26:113
96593199151,6cyclictest0-21swapper/307:21:113
96593199144,6cyclictest0-21swapper/308:40:013
96593199143,5cyclictest0-21swapper/308:41:133
96593199143,5cyclictest0-21swapper/308:41:133
96593199142,8cyclictest0-21swapper/312:35:043
96593199142,8cyclictest0-21swapper/311:06:013
96593199142,6cyclictest0-21swapper/311:40:583
96593199142,5cyclictest0-21swapper/308:46:093
96593199141,9cyclictest0-21swapper/312:21:053
96593199141,8cyclictest0-21swapper/312:11:013
96593199141,8cyclictest0-21swapper/312:11:013
96593199141,8cyclictest0-21swapper/312:00:063
96593199141,8cyclictest0-21swapper/311:46:083
96593199141,8cyclictest0-21swapper/311:16:073
96593199141,8cyclictest0-21swapper/311:11:003
96593199141,8cyclictest0-21swapper/310:41:033
96593199141,8cyclictest0-21swapper/310:31:003
96593199141,8cyclictest0-21swapper/307:46:103
96593199141,7cyclictest0-21swapper/312:26:043
96593199141,7cyclictest0-21swapper/311:55:593
96593199141,7cyclictest0-21swapper/311:26:013
96593199141,7cyclictest0-21swapper/311:26:013
96593199141,7cyclictest0-21swapper/309:25:593
96593199133,7cyclictest0-21swapper/308:50:573
96593199132,7cyclictest0-21swapper/309:01:043
96593199132,6cyclictest0-21swapper/308:11:113
96593199132,5cyclictest0-21swapper/311:50:583
96593199131,8cyclictest0-21swapper/310:51:043
96593199131,8cyclictest0-21swapper/309:31:063
96593199131,7cyclictest0-21swapper/310:20:553
96593199131,5cyclictest0-21swapper/309:06:083
96593199122,6cyclictest0-21swapper/307:40:003
96593199121,6cyclictest0-21swapper/308:56:043
96593199121,6cyclictest0-21swapper/308:06:093
96593199121,6cyclictest0-21swapper/307:56:113
96593199112,6cyclictest0-21swapper/307:51:043
96593199112,5cyclictest0-21swapper/307:11:083
96593199111,6cyclictest0-21swapper/312:16:103
96593199111,6cyclictest0-21swapper/308:20:023
96593199111,6cyclictest0-21swapper/307:20:013
96593199111,5cyclictest0-21swapper/308:26:013
96593199111,5cyclictest0-21swapper/307:40:583
96593199111,5cyclictest0-21swapper/307:10:033
96593199101,5cyclictest0-21swapper/307:31:033
96593199101,5cyclictest0-21swapper/307:31:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional