You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 05:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot7s.osadl.org (updated Wed Apr 15, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2081329970,9sleep20-21swapper/219:08:002
2072429267,9sleep00-21swapper/019:06:500
2065928342,37sleep30-21swapper/319:06:003
2064727857,6sleep10-21swapper/119:05:511
2105899280,25cyclictest0-21swapper/200:35:052
2105899260,23cyclictest0-21swapper/221:27:292
2106199250,1cyclictest0-21swapper/321:38:063
2105899250,24cyclictest0-21swapper/222:28:202
2106199230,2cyclictest0-21swapper/321:48:293
2106199220,2cyclictest0-21swapper/300:15:293
2106199220,0cyclictest0-21swapper/323:56:213
2105599220,18cyclictest0-21swapper/119:30:141
2106199212,11cyclictest7566-21sensors23:00:243
2105599210,21cyclictest0-21swapper/122:30:351
2106199200,1cyclictest0-21swapper/323:38:483
2106199200,0cyclictest0-21swapper/322:57:383
2105899203,2cyclictest455-21dbus-daemon00:07:222
21058992018,1cyclictest14153-21apache_processe21:10:132
2105899200,20cyclictest0-21swapper/222:43:042
2106199190,1cyclictest0-21swapper/321:58:153
2105899190,19cyclictest0-21swapper/223:07:082
2105899190,19cyclictest0-21swapper/200:03:032
2105899190,18cyclictest0-21swapper/223:24:122
2105199194,2cyclictest141rcu_preempt23:23:270
21051991913,6cyclictest13-21ksoftirqd/022:30:150
2106199181,4cyclictest0-21swapper/300:35:133
2106199180,1cyclictest0-21swapper/322:43:403
2106199180,0cyclictest0-21swapper/323:52:183
2106199180,0cyclictest0-21swapper/323:47:153
2105899183,3cyclictest141rcu_preempt22:55:092
2105899181,15cyclictest0-21swapper/222:47:282
2105899180,17cyclictest0-21swapper/223:12:212
2105899180,17cyclictest0-21swapper/222:39:552
2105899180,17cyclictest0-21swapper/200:15:362
21055991816,1cyclictest0-21swapper/123:30:131
21055991814,3cyclictest0-21swapper/123:40:201
2105599180,18cyclictest0-21swapper/123:45:521
21051991817,1cyclictest13-21ksoftirqd/023:05:380
2106199170,1cyclictest0-21swapper/321:22:413
2106199170,1cyclictest0-21swapper/300:14:023
2105899170,17cyclictest0-21swapper/223:15:452
2105899170,14cyclictest0-21swapper/222:00:492
21055991717,0cyclictest0-21swapper/122:11:161
21055991715,2cyclictest0-21swapper/122:09:151
2105599170,17cyclictest0-21swapper/100:13:231
21051991716,1cyclictest0-21swapper/000:37:070
21051991715,1cyclictest7776-1kworker/u9:023:44:010
2105199170,14cyclictest0-21swapper/000:08:510
21061991615,1cyclictest0-21swapper/321:40:503
21061991610,4cyclictest0-21swapper/321:50:163
2106199160,2cyclictest0-21swapper/322:04:163
2106199160,1cyclictest0-21swapper/323:14:293
2106199160,1cyclictest0-21swapper/322:35:503
2106199160,1cyclictest0-21swapper/322:21:163
2106199160,1cyclictest0-21swapper/322:15:203
2105899160,2cyclictest0-21swapper/200:24:052
2105899160,15cyclictest0-21swapper/222:10:582
2105599162,13cyclictest13630-21diskmemload22:28:281
21055991615,1cyclictest13630-21diskmemload21:44:211
2105599160,16cyclictest0-21swapper/122:16:551
2105199164,11cyclictest12077-21runrttasks22:05:420
2105199160,14cyclictest0-21swapper/021:30:080
2106199150,1cyclictest0-21swapper/321:30:333
2106199150,1cyclictest0-21swapper/321:12:233
2106199150,1cyclictest0-21swapper/300:09:183
2105899154,5cyclictest0-21swapper/221:30:112
2105899150,2cyclictest0-21swapper/223:59:282
2105599158,6cyclictest9781-21sh23:59:591
21055991512,3cyclictest0-21swapper/122:03:371
21055991511,3cyclictest25-21ksoftirqd/122:48:281
21051991511,2cyclictest0-21swapper/000:10:130
2105199150,12cyclictest0-21swapper/023:36:350
75412140,0sleep10-21swapper/123:00:241
21061991412,1cyclictest0-21swapper/322:48:253
21061991411,2cyclictest22408-21cat23:10:013
2106199140,2cyclictest0-21swapper/322:11:033
2106199140,0cyclictest0-21swapper/323:19:283
2105899149,2cyclictest22285-21sed22:50:182
2105899147,3cyclictest0-21swapper/221:46:012
21058991411,3cyclictest3561-21apache223:30:092
21058991411,2cyclictest2618-21cut20:45:192
21058991410,3cyclictest4848-21turbostat.cron00:35:002
21058991410,3cyclictest25823-21grep23:50:112
21058991410,2cyclictest0-21swapper/222:30:252
2105899140,14cyclictest0-21swapper/221:23:332
2105599144,5cyclictest38150irq/131-snd_hda23:10:161
2105599144,2cyclictest225-21systemd-journal00:06:111
2105599141,2cyclictest141rcu_preempt00:23:551
21055991411,2cyclictest1515-21cut19:35:171
2105599141,12cyclictest13630-21diskmemload21:57:561
2105599140,13cyclictest0-21swapper/121:32:231
21051991412,2cyclictest13-21ksoftirqd/022:35:050
2105199140,1cyclictest0-21swapper/023:58:080
2106199139,1cyclictest0-21swapper/321:26:303
2106199138,1cyclictest0-21swapper/300:02:333
2106199137,3cyclictest537-21snmpd23:21:453
2106199131,11cyclictest13913-21ssh00:20:243
21061991310,2cyclictest28201-21cat20:30:183
21061991310,2cyclictest22150-21cut22:50:173
21061991310,2cyclictest16485-21grep23:25:163
21061991310,2cyclictest11241-21cron21:05:003
2106199130,3cyclictest0-21swapper/322:34:463
2105899139,3cyclictest6724-21tr19:45:202
2105899139,3cyclictest31745-21uname19:30:192
2105899139,3cyclictest29192-21grep19:25:172
2105899139,3cyclictest14347-21grep20:00:252
2105899136,3cyclictest8324-21cat23:39:592
2105899135,3cyclictest0-21swapper/219:15:152
2105899134,7cyclictest0-21swapper/222:05:202
21058991310,2cyclictest17470-21cut21:50:162
21058991310,2cyclictest13224-21grep21:05:282
21058991310,2cyclictest13224-21grep21:05:272
2105899130,13cyclictest0-21swapper/221:16:112
2105599139,3cyclictest10987-21chrt23:21:481
21055991310,2cyclictest6774-21cat21:25:011
21055991310,2cyclictest30084-21wc22:35:251
21055991310,2cyclictest29159-21taskset19:25:161
21055991310,2cyclictest25304-21grep19:15:281
21055991310,2cyclictest17536-21grep23:25:251
2105599130,11cyclictest0-21swapper/121:51:231
2105199139,3cyclictest23410-21grep22:50:260
2105199133,6cyclictest0-21swapper/000:00:270
2105199131,7cyclictest30659-21munin-run23:15:010
2105199131,6cyclictest2901-21idleruntime-cro19:40:000
21051991310,2cyclictest7042sleep021:59:590
21051991310,2cyclictest25767-21cron19:20:010
2105199130,2cyclictest0-21swapper/023:45:430
2105199130,1cyclictest0-21swapper/022:15:240
2105199130,12cyclictest22761-21tr20:20:150
2105199130,12cyclictest10834-21grep21:00:270
2106199129,2cyclictest29466-21cut00:30:143
2106199129,2cyclictest29200-21sed19:25:173
2106199129,2cyclictest2865-21cron19:40:003
2106199129,2cyclictest26663-21cut19:20:163
2106199129,2cyclictest25601-21sed20:25:173
2106199129,2cyclictest25369-21grep19:15:283
2106199129,2cyclictest23586-21wc19:15:013
2106199129,2cyclictest22272-21cat20:19:593
2106199128,2cyclictest56782sleep320:50:223
2106199123,8cyclictest0-21swapper/322:28:593
2106199120,1cyclictest0-21swapper/323:41:183
2106199120,1cyclictest0-21swapper/322:05:123
2105899129,2cyclictest6780-21sed23:00:172
2105899129,2cyclictest5472-21grep20:50:202
2105899129,2cyclictest30814-21grep20:35:222
2105899129,2cyclictest28041-21expr00:10:162
2105899129,2cyclictest25834-21wc20:25:192
2105899129,2cyclictest22548-21wc19:10:212
2105899129,2cyclictest21564-21uniq00:25:182
2105899129,2cyclictest1237-21tail20:40:262
2105899129,2cyclictest10825-21df19:55:152
2105899129,2cyclictest10400-21wc21:00:232
21058991211,1cyclictest0-21swapper/223:48:502
2105899120,9cyclictest0-21swapper/219:35:132
2105899120,12cyclictest0-21swapper/220:15:142
2105899120,10cyclictest0-21swapper/221:37:372
2105599129,2cyclictest4741-21grep22:20:261
2105599129,2cyclictest23417-21head22:50:261
2105599129,2cyclictest23318-21ssh21:15:181
2105599129,2cyclictest14870-21wc20:04:591
2105599129,2cyclictest13630-21diskmemload22:42:121
2105599129,2cyclictest10618-21tr19:55:141
2105599128,3cyclictest5332-21wc20:50:191
2105599127,4cyclictest26735-21ssh21:36:051
2105599127,4cyclictest16337-21tr20:05:211
2105599126,5cyclictest28305-21cut20:30:181
2105599120,1cyclictest0-21swapper/100:39:251
2105199129,2cyclictest733-21sed23:15:280
2105199129,2cyclictest31749-21sed22:55:260
2105199129,2cyclictest31712-21cat20:40:010
2105199129,2cyclictest23467-21cat19:15:000
2105199129,2cyclictest21657-21tail00:25:170
2105199129,2cyclictest21549-21ssh23:28:250
2105199129,2cyclictest18116-21cut22:10:110
2105199129,2cyclictest12290-21grep22:25:160
2105199123,5cyclictest12433-21cron20:00:000
312092110,1sleep10-21swapper/123:15:151
2106199118,2cyclictest7016-21grep20:55:153
2106199118,2cyclictest6037-21expr19:45:153
2106199118,2cyclictest340-21taskset19:33:453
2106199118,2cyclictest3384-21expr19:40:163
2106199118,2cyclictest31692-21cron20:39:593
2106199118,2cyclictest29774-21ssh23:33:073
2106199118,2cyclictest28482-21ssh21:18:253
2106199118,2cyclictest21106-21tr00:25:153
2106199118,2cyclictest1977-21wget20:45:163
2106199118,2cyclictest12440-21sh20:00:003
2105899118,2cyclictest9497-21cat19:50:242
2105899118,2cyclictest8730-21taskset20:58:362
2105899118,2cyclictest6657-21ssh21:44:032
2105899118,2cyclictest3120-21tr22:20:142
2105899118,2cyclictest27455-21sensors_temp19:20:242
2105899118,2cyclictest26265-21ssh22:15:022
2105899118,2cyclictest16654-21cat23:25:172
2105899118,2cyclictest10501-21ssh23:40:242
2105899117,3cyclictest26082-21sed21:55:172
2105899110,11cyclictest0-21swapper/220:12:452
2105599118,2cyclictest9203-21ssh21:25:311
2105599118,2cyclictest9066-21ssh21:45:171
2105599118,2cyclictest3930-21taskset20:47:541
2105599118,2cyclictest3786-21sh00:15:131
2105599118,2cyclictest32695-21grep20:40:171
2105599118,2cyclictest30270-21cut20:35:181
2105599118,2cyclictest29786-21ssh22:55:111
2105599118,2cyclictest29656-21cstates00:30:151
2105599118,2cyclictest26510-21grep23:50:141
2105599118,2cyclictest25106-21ssh00:27:111
2105599118,2cyclictest1876-21cut23:35:221
2105599118,2cyclictest14954-21grep21:10:171
2105599118,2cyclictest11236-21which00:00:181
2105599117,2cyclictest24569-21cron20:25:001
2105599116,4cyclictest22107-21taskset20:19:101
2105599114,6cyclictest15196-21gltestperf23:05:181
2105599111,9cyclictest38150irq/131-snd_hda19:50:121
2105199118,2cyclictest9595-21chrt21:25:490
2105199118,2cyclictest6210-21ssh21:24:440
2105199118,2cyclictest5588-21ls20:50:210
2105199118,2cyclictest5187-21grep22:40:180
2105199118,2cyclictest4662-21ssh00:15:180
2105199118,2cyclictest4282-21ssh21:42:130
2105199118,2cyclictest4045-21basename19:40:190
2105199118,2cyclictest3972-21cat22:20:190
2105199118,2cyclictest3598-21ssh22:00:410
2105199118,2cyclictest32286-21chrt23:53:570
2105199118,2cyclictest32076-21ssh00:31:010
2105199118,2cyclictest29518-21ssh23:32:540
2105199118,2cyclictest29285-21taskset20:34:370
2105199118,2cyclictest22769-21basename21:15:150
2105199118,2cyclictest21177-21grep20:15:200
2105199118,2cyclictest20873-21sh21:52:040
2105199118,2cyclictest14260-21tr21:10:150
2105199118,2cyclictest13642-21ssh23:04:520
2105199118,2cyclictest13395-21df22:45:160
2105199118,2cyclictest10815-21chrt21:45:450
2105199117,3cyclictest13499-21grep00:20:190
2105199117,3cyclictest13468-21head20:00:160
2105199110,6cyclictest0-21swapper/019:50:120
2106199108,1cyclictest8255-21expr19:50:143
2106199108,1cyclictest22676-21cat20:20:153
2106199108,1cyclictest135892sleep321:09:463
2106199108,1cyclictest135892sleep321:09:453
2106199107,2cyclictest399-21cat20:40:183
2106199107,2cyclictest17451-21basename20:10:103
2106199107,2cyclictest15381-21grep20:05:153
2106199107,2cyclictest13480-21head20:00:163
2105899108,1cyclictest27842-21uniq20:30:162
2105899108,1cyclictest24409-21taskset20:22:582
2105899108,1cyclictest152472sleep220:05:142
2105899107,2cyclictest4431-21pmu-power19:40:222
2105599108,1cyclictest9420-21expr21:00:161
2105599108,1cyclictest7352-21chrt19:45:411
2105599108,1cyclictest50562sleep119:41:411
2105599108,1cyclictest25977-21ntpq20:25:201
2105599108,1cyclictest11887-21ls21:05:171
2105599108,1cyclictest11887-21ls21:05:161
2105599107,2cyclictest8695-21taskset20:58:081
2105599107,2cyclictest22544-21grep19:10:211
2105599107,2cyclictest19474-21taskset20:13:421
2105199108,2cyclictest18620-21awk20:10:200
2105199108,1cyclictest84892sleep020:55:280
2105199108,1cyclictest5782-21cpuspeed_turbos19:45:150
2105199108,1cyclictest25274-21ssh21:35:220
2105199108,1cyclictest150632sleep020:05:030
2105199107,2cyclictest32756-21grep20:40:170
2105199107,2cyclictest28437-21apt19:25:120
2105199107,2cyclictest26291-21grep19:20:140
2105199107,2cyclictest13312-21seq21:06:100
2105199107,2cyclictest13312-21seq21:06:100
2105199102,5cyclictest30813-21apt-get19:30:140
2105199100,1cyclictest0-21swapper/020:25:150
210559997,1cyclictest26808-21tr19:20:171
210519997,1cyclictest2366-21expr20:45:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional