You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 13:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Apr 19, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
105554921450,3sleep20-21swapper/222:55:222
106663921380,10sleep234-21ksoftirqd/223:30:112
106663921380,10sleep234-21ksoftirqd/223:30:112
97389021170,9sleep10-21swapper/119:05:281
100024521030,2sleep13056-21PK-Backend20:20:111
97433228139,14sleep20-21swapper/219:09:562
97360826642,15sleep30-21swapper/319:05:223
97429526241,14sleep00-21swapper/019:09:250
9842162510,2sleep30-21swapper/319:35:143
10422532510,3sleep13056-21PK-Backend22:20:131
10422532510,3sleep13056-21PK-Backend22:20:131
10780112490,3sleep13056-21PK-Backend00:00:211
9854022480,3sleep03056-21PK-Backend19:40:000
10721702480,1sleep20-21swapper/223:45:112
10105852350,11chrt34-21ksoftirqd/220:50:032
10547392330,2sleep10-21swapper/122:55:131
974488992824,3cyclictest411rcuc/319:56:013
974488992724,2cyclictest411rcuc/319:48:003
97448899263,11cyclictest141rcu_preempt19:23:183
974488992623,2cyclictest411rcuc/319:14:593
974486992611,3cyclictest34-21ksoftirqd/223:13:382
10518722260,3chrt0-21swapper/222:45:202
9984082250,3chrt0-21swapper/220:15:152
97448899256,6cyclictest42-21ksoftirqd/322:40:103
974488992522,2cyclictest411rcuc/322:13:193
974488992522,2cyclictest411rcuc/321:31:003
974488992522,2cyclictest411rcuc/321:10:433
974488992522,2cyclictest411rcuc/320:48:313
974488992521,3cyclictest411rcuc/323:04:443
974488992521,3cyclictest411rcuc/322:49:043
974488992521,3cyclictest411rcuc/321:49:253
974488992521,3cyclictest411rcuc/321:43:073
974488992521,3cyclictest411rcuc/320:51:473
974488992521,3cyclictest411rcuc/320:33:263
974488992521,3cyclictest411rcuc/320:12:053
97448699259,3cyclictest34-21ksoftirqd/222:02:172
974488992421,2cyclictest411rcuc/323:55:123
974488992421,2cyclictest411rcuc/323:46:063
974488992421,2cyclictest411rcuc/323:43:493
974488992421,2cyclictest411rcuc/323:30:343
974488992421,2cyclictest411rcuc/323:30:343
974488992421,2cyclictest411rcuc/323:26:413
974488992421,2cyclictest411rcuc/322:57:503
974488992421,2cyclictest411rcuc/322:08:303
974488992421,2cyclictest411rcuc/320:57:003
974488992421,2cyclictest411rcuc/320:42:123
974488992421,2cyclictest411rcuc/319:30:033
974488992421,2cyclictest411rcuc/319:29:343
974488992421,2cyclictest411rcuc/319:19:293
974488992420,3cyclictest411rcuc/323:50:473
974488992420,3cyclictest411rcuc/323:38:123
974488992420,3cyclictest411rcuc/322:29:073
974488992420,3cyclictest411rcuc/322:22:063
974488992420,3cyclictest411rcuc/322:22:053
974488992420,3cyclictest411rcuc/321:26:073
974488992420,3cyclictest411rcuc/321:16:213
974488992420,3cyclictest411rcuc/320:18:563
974488992420,3cyclictest411rcuc/300:24:033
974488992420,3cyclictest411rcuc/300:01:463
97448699249,4cyclictest34-21ksoftirqd/222:26:462
974486992412,3cyclictest34-21ksoftirqd/200:03:342
974486992411,7cyclictest34-21ksoftirqd/223:28:582
974486992410,3cyclictest34-21ksoftirqd/222:14:592
10675782240,3sleep00-21swapper/023:30:210
10675782240,3sleep00-21swapper/023:30:210
97448899235,9cyclictest42-21ksoftirqd/323:20:203
974488992320,2cyclictest411rcuc/323:15:023
974488992320,2cyclictest411rcuc/323:10:053
974488992320,2cyclictest411rcuc/323:06:003
974488992320,2cyclictest411rcuc/322:50:053
974488992320,2cyclictest411rcuc/322:37:173
974488992320,2cyclictest411rcuc/322:31:033
974488992320,2cyclictest411rcuc/322:02:483
974488992320,2cyclictest411rcuc/321:55:433
974488992320,2cyclictest411rcuc/321:35:463
974488992320,2cyclictest411rcuc/321:23:403
974488992320,2cyclictest411rcuc/320:35:473
974488992320,2cyclictest411rcuc/320:20:473
974488992320,2cyclictest411rcuc/320:09:283
974488992320,2cyclictest411rcuc/319:50:033
974488992320,2cyclictest411rcuc/319:41:413
974488992320,2cyclictest411rcuc/300:31:013
974488992320,2cyclictest411rcuc/300:31:013
974488992320,2cyclictest411rcuc/300:28:443
974488992320,2cyclictest411rcuc/300:15:053
974488992320,2cyclictest411rcuc/300:10:083
974488992320,2cyclictest411rcuc/300:09:403
974488992319,3cyclictest411rcuc/322:15:363
974488992319,3cyclictest411rcuc/320:26:413
974488992319,3cyclictest411rcuc/320:00:543
974488992319,3cyclictest411rcuc/300:36:223
97448699238,4cyclictest34-21ksoftirqd/221:20:302
974486992320,2cyclictest1023341-21memory21:25:162
974486992311,3cyclictest34-21ksoftirqd/223:15:302
974486992310,3cyclictest34-21ksoftirqd/222:15:402
974486992310,3cyclictest34-21ksoftirqd/200:15:252
97448399230,21cyclictest3056-21PK-Backend00:27:241
974480992316,6cyclictest1032630-21taskset21:51:520
97448899222,2cyclictest141rcu_preempt21:50:173
974488992219,2cyclictest411rcuc/321:05:463
974488992219,2cyclictest411rcuc/321:00:533
97448699229,4cyclictest34-21ksoftirqd/221:14:282
97448699229,3cyclictest34-21ksoftirqd/220:25:092
974486992210,3cyclictest34-21ksoftirqd/220:14:262
97448699220,20cyclictest1042063-21/usr/sbin/munin22:20:182
97448699220,20cyclictest1042063-21/usr/sbin/munin22:20:182
974483992220,1cyclictest3056-21PK-Backend21:09:331
974483992219,2cyclictest3056-21PK-Backend23:13:021
974483992214,6cyclictest1031148-21cron21:50:001
97448399220,2cyclictest3056-21PK-Backend20:04:371
97448399220,20cyclictest3056-21PK-Backend21:37:171
97448399220,20cyclictest3056-21PK-Backend21:11:351
97448399220,20cyclictest3056-21PK-Backend20:34:441
97448399220,20cyclictest3056-21PK-Backend20:28:011
974480992218,2cyclictest1788620-21WebKitWebProces00:25:560
139099220,4rtkit-daemon1389-21rtkit-daemon20:26:310
97448699219,3cyclictest34-21ksoftirqd/222:09:022
97448699218,3cyclictest34-21ksoftirqd/221:02:012
97448699218,3cyclictest34-21ksoftirqd/219:25:252
97448699218,3cyclictest34-21ksoftirqd/219:16:152
974486992118,2cyclictest1064470-21chrt23:23:092
974486992113,6cyclictest980135-21unin-run19:25:012
974486992111,4cyclictest34-21ksoftirqd/220:05:112
974486992111,3cyclictest34-21ksoftirqd/223:40:372
97448699210,20cyclictest0-21swapper/222:42:012
974483992119,1cyclictest3056-21PK-Backend23:55:261
974483992119,1cyclictest3056-21PK-Backend23:52:501
974483992119,1cyclictest3056-21PK-Backend23:08:491
974483992119,1cyclictest3056-21PK-Backend21:30:271
974483992119,1cyclictest3056-21PK-Backend20:36:481
974483992119,1cyclictest3056-21PK-Backend19:29:551
974483992119,1cyclictest3056-21PK-Backend19:20:561
974483992118,2cyclictest3056-21PK-Backend22:49:361
974483992118,2cyclictest3056-21PK-Backend20:09:291
974483992118,2cyclictest3056-21PK-Backend00:31:021
974483992118,2cyclictest3056-21PK-Backend00:31:021
97448399210,20cyclictest3056-21PK-Backend22:14:551
97448399210,19cyclictest3056-21PK-Backend23:45:121
97448399210,19cyclictest3056-21PK-Backend21:21:491
97448399210,19cyclictest3056-21PK-Backend20:47:081
97448399210,19cyclictest3056-21PK-Backend19:36:521
97448399210,19cyclictest3056-21PK-Backend00:10:541
97448399210,19cyclictest3056-21PK-Backend00:08:171
974480992119,1cyclictest0-21swapper/023:27:290
974480992119,1cyclictest0-21swapper/020:45:190
974480992119,1cyclictest0-21swapper/020:14:180
974480992119,1cyclictest0-21swapper/020:01:460
97448699208,3cyclictest34-21ksoftirqd/220:45:492
97448699206,3cyclictest141rcu_preempt20:22:032
97448699201,8cyclictest34-21ksoftirqd/220:34:302
97448699201,7cyclictest34-21ksoftirqd/221:50:002
97448699201,5cyclictest1054212-21turbostat22:55:002
974486992014,4cyclictest992674-21cut20:00:142
974486992011,3cyclictest34-21ksoftirqd/223:52:402
97448699200,19cyclictest0-21swapper/223:39:202
97448699200,19cyclictest0-21swapper/223:02:172
97448699200,19cyclictest0-21swapper/221:51:042
97448699200,19cyclictest0-21swapper/219:56:542
97448699200,19cyclictest0-21swapper/219:52:492
97448699200,19cyclictest0-21swapper/219:35:382
97448699200,19cyclictest0-21swapper/200:21:582
97448699200,19cyclictest0-21swapper/200:12:192
974483992018,1cyclictest3056-21PK-Backend23:32:021
974483992018,1cyclictest3056-21PK-Backend23:32:011
974483992018,1cyclictest3056-21PK-Backend22:09:071
974483992018,1cyclictest3056-21PK-Backend22:02:181
974483992018,1cyclictest3056-21PK-Backend21:27:501
974483992018,1cyclictest3056-21PK-Backend20:44:171
974483992018,1cyclictest0-21swapper/122:54:361
974483992017,2cyclictest3056-21PK-Backend19:46:511
974483992017,2cyclictest3056-21PK-Backend19:15:011
974483992013,5cyclictest1045539-21cron22:29:591
97448399200,19cyclictest3056-21PK-Backend20:14:231
97448399200,19cyclictest3056-21PK-Backend19:51:451
97448399200,19cyclictest3056-21PK-Backend19:44:491
97448399200,19cyclictest0-21swapper/121:55:011
97448399200,19cyclictest0-21swapper/121:53:121
97448399200,18cyclictest3056-21PK-Backend20:51:001
97448399200,18cyclictest3056-21PK-Backend00:16:561
974480992018,1cyclictest3056-21PK-Backend20:15:180
974480992018,1cyclictest0-21swapper/022:42:130
974480992018,1cyclictest0-21swapper/021:49:140
974480992018,1cyclictest0-21swapper/021:12:240
974480992018,1cyclictest0-21swapper/020:37:280
974480992018,1cyclictest0-21swapper/019:43:350
974480992018,1cyclictest0-21swapper/019:20:090
974480992011,7cyclictest1021492-21cat21:20:160
97448099200,1cyclictest3056-21PK-Backend22:30:170
97448099200,1cyclictest0-21swapper/023:24:180
97448099200,1cyclictest0-21swapper/023:07:010
97448099200,19cyclictest0-21swapper/022:00:160
97448099200,19cyclictest0-21swapper/019:13:550
97448699199,2cyclictest141rcu_preempt00:37:272
97448699194,8cyclictest34-21ksoftirqd/223:07:442
974486991917,1cyclictest0-21swapper/223:57:252
974486991917,1cyclictest0-21swapper/219:33:352
974486991917,1cyclictest0-21swapper/219:11:392
974486991913,4cyclictest2156-21gnome-shell21:57:092
974486991912,5cyclictest988824-21cat19:50:002
974486991912,5cyclictest1048830-21sshd22:37:552
974486991911,6cyclictest1012352-21awk20:55:132
974486991911,6cyclictest1005830-21grep20:35:192
97448699190,1cyclictest0-21swapper/222:31:342
97448699190,1cyclictest0-21swapper/221:33:072
97448699190,1cyclictest0-21swapper/221:08:052
974483991917,1cyclictest3056-21PK-Backend22:17:331
974483991917,1cyclictest3056-21PK-Backend19:12:041
974483991916,2cyclictest3056-21PK-Backend23:39:241
974483991916,2cyclictest3056-21PK-Backend21:41:441
974483991911,6cyclictest1064636-21/usr/sbin/munin23:25:001
97448399190,1cyclictest3056-21PK-Backend23:26:131
97448399190,1cyclictest3056-21PK-Backend22:40:371
97448399190,1cyclictest3056-21PK-Backend22:38:001
97448399190,1cyclictest3056-21PK-Backend19:59:421
974480991917,1cyclictest0-21swapper/020:05:240
974480991914,4cyclictest13-21ksoftirqd/022:25:010
974480991914,4cyclictest13-21ksoftirqd/022:25:010
974480991912,6cyclictest1083745-21cron00:20:000
97448099190,1cyclictest0-21swapper/023:10:120
97448099190,1cyclictest0-21swapper/022:25:300
97448099190,1cyclictest0-21swapper/019:17:590
97448099190,18cyclictest0-21swapper/020:58:370
97448099190,18cyclictest0-21swapper/019:50:510
97448099190,18cyclictest0-21swapper/000:02:370
97448699188,2cyclictest141rcu_preempt00:29:452
974486991816,1cyclictest0-21swapper/221:41:112
974486991811,5cyclictest1079509-21ntpq00:05:172
974486991810,6cyclictest1087197-21apt-get00:30:032
974486991810,6cyclictest1087197-21apt-get00:30:032
97448699180,9cyclictest34-21ksoftirqd/219:40:292
974483991816,1cyclictest3056-21PK-Backend23:04:371
974483991816,1cyclictest3056-21PK-Backend21:17:361
97448399180,1cyclictest3056-21PK-Backend19:30:571
974480991816,1cyclictest0-21swapper/021:04:050
974480991816,1cyclictest0-21swapper/020:43:300
974480991816,1cyclictest0-21swapper/019:31:520
974480991816,1cyclictest0-21swapper/019:28:130
974480991813,3cyclictest1038357-21sh22:10:100
974480991812,5cyclictest883-21dbus-daemon00:40:000
974480991811,6cyclictest883-21dbus-daemon00:25:000
974480991811,5cyclictest1016360-21wc21:05:170
97448099180,16cyclictest988776-21perf19:50:000
974486991713,3cyclictest34-21ksoftirqd/221:40:002
974486991712,3cyclictest1007927-21timerwakeupswit20:40:212
974483991715,1cyclictest3056-21PK-Backend21:03:371
974483991715,1cyclictest3056-21PK-Backend20:18:281
974483991714,2cyclictest3056-21PK-Backend00:22:101
97448099179,6cyclictest1037351-21awk22:05:190
97448099174,12cyclictest1061406-21sort23:15:110
974480991715,1cyclictest0-21swapper/023:35:130
974480991715,1cyclictest0-21swapper/022:19:560
974480991715,1cyclictest0-21swapper/021:35:280
974480991715,1cyclictest0-21swapper/021:30:420
974480991715,1cyclictest0-21swapper/020:52:220
974480991712,4cyclictest1000514-21sed20:20:150
974480991711,4cyclictest13-21ksoftirqd/023:51:480
974480991711,4cyclictest1081373-21cut00:10:200
97448099170,16cyclictest0-21swapper/022:47:350
97448099170,16cyclictest0-21swapper/022:38:140
97448699161,9cyclictest34-21ksoftirqd/221:15:052
97448399169,5cyclictest1046391-21load22:30:151
974483991614,1cyclictest3056-21PK-Backend00:39:071
974483991614,1cyclictest0-21swapper/123:17:141
97448099168,6cyclictest1089212-21munin-run00:35:000
97448099168,6cyclictest1089212-21munin-run00:35:000
97448099168,6cyclictest1054840-21df22:55:140
974480991614,1cyclictest0-21swapper/023:49:050
974480991614,1cyclictest0-21swapper/019:58:080
974480991612,3cyclictest1070076-21/usr/sbin/munin23:40:210
974480991612,3cyclictest1019271-21/usr/sbin/munin21:15:130
974480991611,3cyclictest1052891-21tr22:50:110
974480991610,5cyclictest883-21dbus-daemon23:01:450
97448099160,15cyclictest0-21swapper/023:59:260
974483991512,2cyclictest3056-21PK-Backend23:44:251
97448099159,4cyclictest1023411-21tr21:25:160
97448099158,5cyclictest883-21dbus-daemon22:00:000
97448099150,14cyclictest0-21swapper/020:33:500
974483991412,1cyclictest3056-21PK-Backend20:55:531
97448099148,4cyclictest1078913-21wc00:05:130
974480991310,2cyclictest1028326-21df21:40:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional